Partial message encryption in BizTalk pipelines

If you are receiving messages which contain potentially sensitive data, for example credit card details or bank identifier codes, then you may not want to allow these to enter BizTalk unencrypted as there will be plain-text copies of the data in the message box database. On the other hand, you don't want to encrypt the entire message as its type and at least some of the content are probably needed for routing.

What would be really useful in this circumstance is a pipeline component which can encrypt the sensitive parts of the message as indicated by XPath locations in a receive pipeline, and then transparently replace the encrypted data with the original data in a send pipeline. This would let us flow the information through without it ever being persisted in plain text. Oh, and of course this would need to be done in a streaming manner for speed and memory efficiency.

BizTalk has some very useful (though admittedly undocumented) classes in the Microsoft.BizTalk.Streaming assembly, and for this type of pipeline component the XPathMutatorStream makes the implementation almost trivial. It allows a number of XPath locations to be specified, and each time a location match is found as the message is read, it calls a user-supplied callback method and allows you to replace the value of the location.

I won't go into all of the boiler plate code for the pipeline components, but here's the guts of a simple implementation which allows the name of a symmetric algorithm to be supplied in its configuration properties, an array of XPath locations to encrypt, and whether the mode it is currently working in is encryption or decryption. In this case I've just stored the key and IV for a symmetric encryption algorithm in files (which can be ACL'd to prevent unauthorised access) and allowed the paths to be specified, though you could use any key storage mechanism you want. Note that I have stripped all validation and instrumentation out to make the code clearer.

Below is the Execute method which is called for each message entering the pipeline. It initialises an instance of the encryption algorithm and replaces the body stream of the message with an XPathMutatorStream. The CloneMessage function is a common one which clones all of the message parts and context except the body and creates a new body part; although you can just set the body data on the input message I believe this is a more correct approach because messages should be considered immutable within BizTalk.

public override IBaseMessage Execute(
    IPipelineContext context, 
    IBaseMessage inputMessage)
{
    // create the algorithm instance
    if (this.algorithmInstance == null)
    {
        this.algorithmInstance = SymmetricAlgorithm.Create(this.algorithm);
        this.algorithmInstance.IV = File.ReadAllBytes(this.ivFile);
        this.algorithmInstance.Key = File.ReadAllBytes(this.keyFile);
    }

    // create a collection of body xpaths to fire events on
    XPathCollection xpaths = new XPathCollection();
    foreach (string fieldXPath in this.fieldXPaths)
    {
        xpaths.Add(fieldXPath);
    }

    // return a cloned message with the xpath mutator body stream
    ValueMutator callback = new ValueMutator(this.OnXPathMatch);
    Stream newBodyDataStream = new XPathMutatorStream(
        inputMessage.BodyPart.GetOriginalDataStream(), 
        xpaths, 
        callback);
    IBaseMessage outputMessage = CloneMessage(
        context.GetMessageFactory(), 
        inputMessage, 
        newBodyDataStream);
    return outputMessage;
}

The function that actually does the work is OnXPathMatch, which based on the encryption mode of the component either converts the value of the field from plain text to an encrypted base-64 string, or vice versa. Unicode is chosen as the encoding for conversion to/from byte arrays as this is the internal representation of strings within the .NET Framework irrespective of the original encoding of the message.

private void OnXPathMatch(
    int matchIdx, 
    XPathExpression matchExpr, 
    string origVal, 
    ref string finalVal)
{
    if (this.mode == EncryptionMode.Encrypt)
    {
        byte[] originalBytes = Encoding.Unicode.GetBytes(origVal);
        using (ICryptoTransform transform = this.algorithmInstance.CreateEncryptor())
        {
            byte[] finalBytes = transform.TransformFinalBlock(
                originalBytes, 
                0, 
                originalBytes.Length);
            finalVal = Convert.ToBase64String(finalBytes);
        }
    }
    else
    {
        byte[] originalBytes = Convert.FromBase64String(origVal);
        using (ICryptoTransform transform = this.algorithmInstance.CreateDecryptor())
        {
            byte[] finalBytes = transform.TransformFinalBlock(
                originalBytes, 
                0, 
                originalBytes.Length);
            finalVal = Encoding.Unicode.GetString(finalBytes);
        }
    }
}

And just to make sure the listing is complete enough to allow you to implement it within the common boiler plate wrapper, here's the code for the functions that clone the message, which I tend to move to a seperate utility class for use from any custom component.

public static IBaseMessage CloneMessage(
    IBaseMessageFactory factory, 
    IBaseMessage source, 
    Stream newBodyDataStream)
{
    // create new message and copy context and all non-body parts 
    IBaseMessage destination = factory.CreateMessage();
    destination.Context = source.Context;
    CopyAllNonBodyMessageParts(source, destination);

    // create new body part and add it to the destination message
    IBaseMessagePart body = factory.CreateMessagePart();
    body.PartProperties = source.BodyPart.PartProperties;
    body.Data = newBodyDataStream;
    destination.AddPart(source.BodyPartName, body, true);
    return destination;
}

private static void CopyAllNonBodyMessageParts(
    IBaseMessage source, 
    IBaseMessage destination)
{
    for (int i = 0; i < source.PartCount; i++)
    {
        string partName;
        IBaseMessagePart part = source.GetPartByIndex(i, out partName);
        if (partName != source.BodyPartName)
        {
            destination.AddPart(partName, part, false);
        }
    }
}

This demonstrates one of the reasons that BizTalk is such a great platform for messaging solutions. It's highly extensible and if you explore the libraries that come with the product, you can implement very powerful and efficient features with a minimum of effort - other than the boiler plate code and common functions, it was only about 20 lines of code to allow encryption and decryption of arbitrary locations in a message!