Using Attachments

[pacmano]

Attachments are a way to optimize how Connect handles large messages.

Using Attachments reduces the amount of data being stored in the database. If you look at a message in the message browser you can see that in the raw, transformed, and encoded tabs there are message blobs being displayed. Each one of those represents an entry in the database. Using message attachments, you can reduce the amount of data being stored substantially since, generally, attachments only need to be stored once. Afterwards, they are referenced by their ID.

Contents

• Attaching the Entire Message
• Attachments Using Regex
• Attachments Using JavaScript
• Attachments Using DICOM
• Attachments Using Custom Attachment Handler
• Manipulating Attachments Within a Connector
• Removing Attachments
• Cross-Channel Attachments
• Key Facts

Attaching the Entire Message

One of the simplest ways to use Attachments in Connect is simply attaching the entire message. For example, let’s say you have an HTTP Listener that you know will be taking PDF files. If you were to set up your Listener like this:

Then you would see the following in your messages:

But if you were to configure your channel to store the entire message as an Attachment:

Then you would see the following:

As you can see, the PDF bytes are no longer being stored in the message and have been extracted even before the preprocessor.

Attachments Using Regex

Sometimes you may need to work with large blobs of data that are included as part of a message but may not be the entire message. For example, what if an image is embedded in an HL7 message? This is a good scenario for using the Regex Attachment option.

In this example, an image is embedded in OBX 5.5 and we’ll be converting that portion to an Attachment and writing the image to the filesystem.

In the source connector, select “Regex” in the Attachment dropdown and enter your regular expression.

Regex Snippet:

(?:OBX\|(?:[^|]*?\|){4}(?:[^|^]*?\^){4})([^|^]*)[|^]

Now, add a destination File Writer. In the destination, add a Transformer Mapper step to map OBX 5.5 to attachmentId.

This will allow us to get the image by attachment ID and write it. Be sure to check File Type: Binary. Lastly, set the File Writer destination template to: {attachmentId}. This will be replaced with the actual attachmentId which will then be replaced with the attachment itself.

Now save and deploy the channel and send a test message. (You can test with the test_message.txt attached below which contains a Base 64 encoded PNG test image). If all goes well, you should see an image in the directory you specified.

Attachments Using JavaScript

You can also manipulate attachments with JavaScript. Using the same Attachment dropdown select “JavaScript” then click Properties. Add the following to extract the message into an Attachment, add the word “world” to it, and change the message to the attachment id.

JavaScript Snippet:

var attachment = addAttachment(message + " world", "text/plain", false);
return attachment.getAttachmentId();

Now, if you send a test message with the word “hello”, whatever receives it will get “hello world”. Remember, before the destination sends the message out, it will replace the attachment id with the attachment.

Attachments Using DICOM

Some of the larger messages coming through Connect are of the DICOM type. A lot of performance testing and tweaking has gone in to making DICOM processing as efficient as possible but it’s still important that you configure your channels properly to take advantage of these improvements.

It’s best to store DICOM messages as attachments and to ensure that Connect does this, you’ll want to set your source data type to “DICOM” which automatically selects the DICOM attachment type as well. If you, instead, used the “RAW” data type, then your large DICOM messages would be copied several times throughout the message processing pipeline taking up a lot more room and using more memory.

Attachments Using Custom Attachment Handler

For the intrepid Java users out there, it’s also possible to write your own Java class to parse and manipulate attachments. Note, the class must extend MirthAttachmentHandlerProvider.

With a custom attachment handler, the sky is the limit. For instance, this image shows an example custom attachment handler I’ve created and imported as a library. This uses a regular expression to parse the image and also flip the image vertically before attaching.

And the outcome:

Manipulating Attachments Within a Connector

Working with attachments from within your connectors is also available. Let’s say you have a channel already set up to parse out an attachment, but you want to edit that attachment before sending it out. That’s totally doable and we’ve added some new JavaScript methods to assist you with that.

You could do the following from inside a source transformer.

JavaScript Snippet:

// Get a list of the attachments associated with this message
var attachmentList = getAttachments(false)

// From the list of attachments, get the first one
var attachment = attachmentList.get(0);

// Get the content of the attachment
var newString = attachment.getContentString();

// Append to the attachment content
newString = newString + "*Some Additional Content*";

// Write the new content back to the attachment
attachment.setContentString(newString);

// Update the attachment
updateAttachment(attachment, false);

Another way you could do this (possibly decreasing memory consumption):

// Get a list of the Ids of the attachments associated with this message
var attachmentIdList = getAttachmentIds();

// From the list of attachment Ids, get the first one
var attachmentId = attachmentIdList.get(0);

// Get the attachment associated with the Id
var attachment = getAttachment(attachmentId, false);

// Get the content of the attachment
var newString = attachment.getContentString();

// Append to the attachment content
newString = newString + "*Some Additional Content*";

// Write the new content back to the attachment
attachment.setContentString(newString);

// Update the attachment
updateAttachment(attachment, false);

Note, there are also user utility methods that allow you to retrieve attachments from other completely unrelated channels. As long as you know the channel id and message id you can obtain the related attachments.

Here are what a few of those utility methods look like:

getAttachment(channelId, messageId, attachmentId, base64Decode);
getAttachmentIds(channelId, messageId);
updateAttachment(channelId, messageId, attachmentId, data, type, base64Encode);
updateAttachment(channelId, messageId, attachment, base64Encode);

Removing Attachments

Note, for all attachment types, if your channel and subsequent destination does not care about the attachment data at all, you can further reduce disk usage by unchecking the “Store Attachments” checkbox on the Channel configuration screen which will extract the attachment from you message as usual, but will not store the attachment. It’s important to note that this means the attachment data can not be retrieved.

Cross-Channel Attachments

What if you have a Connect channel with an attachment that sends to another channel as part of your message processing pipeline? In this case, the attachment will be extracted and saved, then reattached before sending to the second channel which will then extract and save the attachment before reattaching and sending the message out. This is a sub-optimal situation since the attachment is being extracted and saved twice, but it can be improved.

When setting the “Reattach Attachments” radio button to “No”, the attachment will not be re-attached before sending to the destination. Instead, any attachments found in the message will be expanded to include the Channel id and Message id in addition to the normal attachment id, which allows a downstream channel to properly handle the attachments it receives.

The downstream channel will now receive the attachment, already extracted from the upstream channel meaning the upstream channel is no longer re-attaching and the downstream channel is no longer re-extracting, thus saving time and disk space.

Key Facts

• Attachments are extracted from the message before the preprocessor.
• Attachment data is replaced with an attachment ID.

Further Discussion

You can discuss feature requests in the issues section of the Connect GitHub repository.

[pacmano]

Thanks for the article. (just a test comment here)