No announcement yet.

filters before transformers? or, deleting segments?

  • Filter
  • Time
  • Show
Clear All
new posts

  • filters before transformers? or, deleting segments?

    I'm working on a project where our notifications arrive via delimited text files.

    I need to route the message based on the file contents. I know how to do that; I have a filter working already.

    But, in order for the filter to work, I basically have to parse the file contents.
    Which I'm doing again in a transformer in the secondary channel anyway.

    My first thought was, OK, I'll add a transformer. But, based on what I've seen here and the tutorials, I would guess that filters execute before transformers, in which case this would not work ... is that correct?

    My second thought was, I could just fold most or all of the transformer into the filter, i.e. alter the message contents in the filter, before calling router.routeMessage() (if I decide to call it, that is).
    And I can in fact get the new / transformed information into the message. However, I don't seem to be able to remove the 'delimited' segment.

    I'm sure I can find a workaround ... but I'd like to do this correctly. And parsing the file contents twice (even if the files are small) is certainly less than optimal, so I'm not starting out well

  • #2
    Is there a reason you can't transform the filter at source?

    If you can't, you could try setting a channelMap variable for the message based on whatever rules and filter based on that...something like this:

    In the source transformer (I'm using MSH.10...use whatever's appropriate)

    In your destination filter (where you want to filter this message):
    if (${msg['MSH']['MSH.10']['MSH.10.1'].toString() == 'True') {
    return false;
    } else {
    return true;
    I can be reached through gmail and Google Talk using davidrothbauer at gmail dot com

    Test all my code suggestions prior to implementation


    • #3
      OK, that probably wasn't very clear.

      I guess what I'm asking is:
      1. How do I go about altering, or preferably replacing, the message contents for downstream consumers?
      (I do know how to store things into the channel map; but that won't work if I'm routing the message to a separate channel.)

      2. Is it possible to get a transformer to execute before a filter?



      • #4

        I had this problem when I first started with Mirth. I didn't realize that there were filters and transformers on the source tab.

        So you do your transformations in the source tab and they will carry through to your destinations.
        That is the only way to transform a message prior to your destination filters.

        Things to keep in mind when doing this:

        Source transformations will carry through to every destination in the channel.

        If you have several destinations and need to the message to arrive intact at any of them you need to use the channelMap to determine whether a message should be filtered or not.
        I can be reached through gmail and Google Talk using davidrothbauer at gmail dot com

        Test all my code suggestions prior to implementation


        • #5
          What I'd like to do is take the input i.e.
          and transform it into something like

          ... so that downstream I can just iterate over the message elements without needing to care about what kind of message it is. (At some point I will need to care, but I'm trying to keep my JavaScript code as granular and generic as possible.)

          But, in order to do that, I want to replace (not augment) the message content ... otherwise I will have to have extra code downstream to deal with (ignore) the "delimited" segment/element downstream.

          I can add stuff to the message just fine; that's not the problem. What I want to do is remove the 'delimited' segment. Or, alternatively, replace the message object entirely.

          A secondary --and somewhat separate -- concern is that, if possible, I'd like to avoid parsing the message twice.

          My guess would be I'm missing something really obvious ... but then again I'm not claiming to be anything other than a n00b


          • #6

            In my case it was the opposite ... I knew you could do transformers & filters on the source connector, but I had forgotten you could do them on destinations as well. So I can put a transformer on the source connector, and the routing filter on the destination side ... that should take care of the order-of-execution issue. Thanks!

            Do you by any chance have any nuggets of wisdom concerning the transformation?


            • #7

              Would you consider using a system of router channels? Or what Mirth Corp calls chaining? I've worked with this at large corp sites. It offers decoupling that allows for a lot of varied manipulations.


              • #8
                I believe that's what I'm doing, if I remember the introductory material correctly. (I'm calling router.routeMessage() to invoke another channel.)

                My question was about manipulating the "msg" data structure (and secondarily about doing so efficiently).


                • #9
                  I believe I've discovered at least part of the issue.

                  Since my original source data is delimited text, the 'delimited' element is the root node in the XML representation of the message, and I can't get rid of it.

                  However, after I create the additional segment(s) in the structure I want, I *can* get rid of the (one and only) 'row' element/segment.

                  I should have figured this out sooner, based on the fact(s) that (1) other segments I created showed up "inside" the 'delimited' element, and (2) I didn't need to use 'delimited' as a segment name when reading the message fields (i.e. you use msg['foo'] rather than msg['delimited']['foo']).
                  Last edited by DavidL; 11-16-2011, 09:37 AM. Reason: fixed quoting


                  • #10
                    What method are you using to pick up the messages?

                    The fact that you have to deal with XML is a bit confusing.

                    Can you post a sample message?
                    I can be reached through gmail and Google Talk using davidrothbauer at gmail dot com

                    Test all my code suggestions prior to implementation


                    • #11
                      Is there a reason that you can't use one of your XML files as a template? You can use one in the source filter and any transformer (source or destination) steps. Then it's just a drag and drop process to create a simple filter in Rule Builder or use JavaScript. The same can be done in a transformer.

                      In your destinations you can use an XML file formatted like the one you want to deliver as a template in the "Outbound Message Templates" to again make the process drag and drop. Now you can use the Message Builder or JavaScript to transform you message as you wish.

                      The router I was referring to is the Message Router pattern. It's implemented in Mirth with channel writers and listeners. It spreads all the operations out if you have multiple filter/transformer requirements. The router Java object doesn't offer the same visual pattern representation or opportunity for transformations.


                      • #12
                        I'm just showing the XML because that's what I see from msg.toString(). I think what I have now will work, thanks.

                        I've discovered, however, that when routing to a separate channel I seem to lose some metadata, e.g. in the destination channel dates seem to have been transformed to strings.
                        I don't know for sure that this is going to be an issue. If it does, I'll probably just serialize the message payload as JSON and deserialize it in the destination channel.