Announcement

Collapse
No announcement yet.

Using Attachments

Collapse
X
Collapse
  •  

  • Using Attachments

    Click image for larger version  Name:	PaperClips.jpg Views:	0 Size:	65.9 KB ID:	175056


    Attachments are a way to optimize how Connect handles large messages.

    Using Attachments reduces the amount of data being stored in the database. If you look at a message in the message browser you can see that in the raw, transformed, and encoded tabs there are message blobs being displayed. Each one of those represents an entry in the database. Using message attachments, you can reduce the amount of data being stored substantially since, generally, attachments only need to be stored once. Afterwards, they are referenced by their ID.

    Contents
    • Attaching the Entire Message
    • Attachments Using Regex
    • Attachments Using JavaScript
    • Attachments Using DICOM
    • Attachments Using Custom Attachment Handler
    • Manipulating Attachments Within a Connector
    • Removing Attachments
    • Cross-Channel Attachments
    • Key Facts
    Attaching the Entire Message

    One of the simplest ways to use Attachments in Connect is simply attaching the entire message. For example, let’s say you have an HTTP Listener that you know will be taking PDF files. If you were to set up your Listener like this:

    Click image for larger version  Name:	NoAttachment1.png Views:	0 Size:	75.9 KB ID:	175088


    Then you would see the following in your messages:

    Click image for larger version  Name:	NoAttachment2.png Views:	1 Size:	113.5 KB ID:	175089


    But if you were to configure your channel to store the entire message as an Attachment:

    Click image for larger version  Name:	EntireMessage1.png Views:	1 Size:	172.9 KB ID:	175090


    Then you would see the following:

    Click image for larger version  Name:	EntireMessage2.png Views:	1 Size:	52.5 KB ID:	175091


    As you can see, the PDF bytes are no longer being stored in the message and have been extracted even before the preprocessor.

    Attachments Using Regex

    Sometimes you may need to work with large blobs of data that are included as part of a message but may not be the entire message. For example, what if an image is embedded in an HL7 message? This is a good scenario for using the Regex Attachment option.

    In this example, an image is embedded in OBX 5.5 and we’ll be converting that portion to an Attachment and writing the image to the filesystem.

    In the source connector, select “Regex” in the Attachment dropdown and enter your regular expression.

    Click image for larger version  Name:	RegexAttachment.png Views:	0 Size:	109.6 KB ID:	175095


    Regex Snippet:

    Code:
    (?:OBX\|(?:[^|]*?\|){4}(?:[^|^]*?\^){4})([^|^]*)[|^]
    Now, add a destination File Writer. In the destination, add a Transformer Mapper step to map OBX 5.5 to attachmentId.

    Click image for larger version  Name:	RegexTransformer.png Views:	0 Size:	66.0 KB ID:	175096


    This will allow us to get the image by attachment ID and write it. Be sure to check File Type: Binary. Lastly, set the File Writer destination template to: {attachmentId}. This will be replaced with the actual attachmentId which will then be replaced with the attachment itself.

    Click image for larger version  Name:	FileWriter.png Views:	0 Size:	274.3 KB ID:	175097


    Now save and deploy the channel and send a test message. (You can test with the test_message.txt attached below which contains a Base 64 encoded PNG test image). If all goes well, you should see an image in the directory you specified.

    Attachments Using JavaScript

    You can also manipulate attachments with JavaScript. Using the same Attachment dropdown select “JavaScript” then click Properties. Add the following to extract the message into an Attachment, add the word “world” to it, and change the message to the attachment id.

    Click image for larger version  Name:	JSAttachment.png Views:	0 Size:	72.6 KB ID:	175098


    JavaScript Snippet:

    Code:
    var attachment = addAttachment(message + " world", "text/plain", false);
    return attachment.getAttachmentId();
    Now, if you send a test message with the word “hello”, whatever receives it will get “hello world”. Remember, before the destination sends the message out, it will replace the attachment id with the attachment.

    Attachments Using DICOM

    Some of the larger messages coming through Connect are of the DICOM type. A lot of performance testing and tweaking has gone in to making DICOM processing as efficient as possible but it’s still important that you configure your channels properly to take advantage of these improvements.

    It’s best to store DICOM messages as attachments and to ensure that Connect does this, you’ll want to set your source data type to “DICOM” which automatically selects the DICOM attachment type as well. If you, instead, used the “RAW” data type, then your large DICOM messages would be copied several times throughout the message processing pipeline taking up a lot more room and using more memory.

    Click image for larger version  Name:	DICOMAttachment.png Views:	0 Size:	221.6 KB ID:	175099


    Attachments Using Custom Attachment Handler

    For the intrepid Java users out there, it’s also possible to write your own Java class to parse and manipulate attachments. Note, the class must extend MirthAttachmentHandlerProvider.

    With a custom attachment handler, the sky is the limit. For instance, this image shows an example custom attachment handler I’ve created and imported as a library. This uses a regular expression to parse the image and also flip the image vertically before attaching.

    Click image for larger version  Name:	CustomAttachment.png Views:	0 Size:	285.7 KB ID:	175100


    And the outcome:

    Click image for larger version  Name:	CustomAttachmentFlipped.png Views:	0 Size:	461.4 KB ID:	175101


    Manipulating Attachments Within a Connector

    Working with attachments from within your connectors is also available. Let’s say you have a channel already set up to parse out an attachment, but you want to edit that attachment before sending it out. That’s totally doable and we’ve added some new JavaScript methods to assist you with that.

    You could do the following from inside a source transformer.

    Click image for larger version  Name:	JSManipulateAttachment.png Views:	0 Size:	148.8 KB ID:	175102


    JavaScript Snippet:

    Code:
    // Get a list of the attachments associated with this message
    var attachmentList = getAttachments(false)
    
    // From the list of attachments, get the first one
    var attachment = attachmentList.get(0);
    
    // Get the content of the attachment
    var newString = attachment.getContentString();
    
    // Append to the attachment content
    newString = newString + "*Some Additional Content*";
    
    // Write the new content back to the attachment
    attachment.setContentString(newString);
    
    // Update the attachment
    updateAttachment(attachment, false);
    Another way you could do this (possibly decreasing memory consumption):

    Code:
    // Get a list of the Ids of the attachments associated with this message
    var attachmentIdList = getAttachmentIds();
    
    // From the list of attachment Ids, get the first one
    var attachmentId = attachmentIdList.get(0);
    
    // Get the attachment associated with the Id
    var attachment = getAttachment(attachmentId, false);
    
    // Get the content of the attachment
    var newString = attachment.getContentString();
    
    // Append to the attachment content
    newString = newString + "*Some Additional Content*";
    
    // Write the new content back to the attachment
    attachment.setContentString(newString);
    
    // Update the attachment
    updateAttachment(attachment, false);
    Note, there are also user utility methods that allow you to retrieve attachments from other completely unrelated channels. As long as you know the channel id and message id you can obtain the related attachments.

    Here are what a few of those utility methods look like:

    Code:
    getAttachment(channelId, messageId, attachmentId, base64Decode);
    getAttachmentIds(channelId, messageId);
    updateAttachment(channelId, messageId, attachmentId, data, type, base64Encode);
    updateAttachment(channelId, messageId, attachment, base64Encode);
    Removing Attachments

    Note, for all attachment types, if your channel and subsequent destination does not care about the attachment data at all, you can further reduce disk usage by unchecking the “Store Attachments” checkbox on the Channel configuration screen which will extract the attachment from you message as usual, but will not store the attachment. It’s important to note that this means the attachment data can not be retrieved.

    Click image for larger version  Name:	RemoveAttachments.png Views:	0 Size:	58.2 KB ID:	175103


    Cross-Channel Attachments

    What if you have a Connect channel with an attachment that sends to another channel as part of your message processing pipeline? In this case, the attachment will be extracted and saved, then reattached before sending to the second channel which will then extract and save the attachment before reattaching and sending the message out. This is a sub-optimal situation since the attachment is being extracted and saved twice, but it can be improved.

    When setting the “Reattach Attachments” radio button to “No”, the attachment will not be re-attached before sending to the destination. Instead, any attachments found in the message will be expanded to include the Channel id and Message id in addition to the normal attachment id, which allows a downstream channel to properly handle the attachments it receives.

    Click image for larger version  Name:	ReattachAttachments.png Views:	0 Size:	113.8 KB ID:	175104


    The downstream channel will now receive the attachment, already extracted from the upstream channel meaning the upstream channel is no longer re-attaching and the downstream channel is no longer re-extracting, thus saving time and disk space.

    Click image for larger version  Name:	ReattachAttachmentsId.png Views:	0 Size:	53.0 KB ID:	175105


    Key Facts
    • Attachments are extracted from the message before the preprocessor.
    • Attachment data is replaced with an attachment ID.
    Further Discussion

    You can discuss feature requests in the issues section of the Connect GitHub repository.
    Attached Files
    Last edited by cturczynskyj; 07-28-2020, 12:37 PM.

    • pacmano
      #1
      pacmano commented
      Editing a comment
      Thanks for the article. (just a test comment here)
    Posting comments is disabled.

Categories

Collapse

article_tags

Collapse

Latest Articles

Collapse

  • Using Attachments
    by cturczynskyj


    Attachments are a way to optimize how Connect handles large messages.

    Using Attachments reduces the amount of data being stored in the database. If you look at a message in the message browser you can see that in the raw, transformed, and encoded tabs there are message blobs being displayed. Each one of those represents an entry in the database. Using message attachments, you can reduce the amount of data being stored substantially since, generally, attachments only need...
    07-24-2020, 04:01 PM
  • Using Connect With Docker
    by cturczynskyj



    ​We’re proud to announce that NextGen Connect is now available for use on Docker Hub!

    In the past, NextGen Connect has been available to install and run locally in Windows, Linux, Mac OSX, and our hard and soft Appliance solutions. The developers at NextGen evaluate and use a variety of tools when updating, testing, and debugging Connect. We’ve actually been using Docker for testing Connect for about a year and have found it very convenient and we hope others...
    07-23-2020, 08:45 AM
Working...
X