Announcement

Collapse
No announcement yet.

Immediate HL7 ack after basic validation and filtering on source transformer side

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Immediate HL7 ack after basic validation and filtering on source transformer side

    Hi,
    I have a channel that converts from HL7 to FHIR as attached . It has some filtering logic in the source where certain messageTypes and subTypes are filtered based on some pre-configured values in deploy script. I am unable to figure out a setup where the response acknowledgement is returned after the processing on the source side (i.e. execution of the filters on the source side). I tried the following combinations :
    1) Source queue ( ON) - Response - Auto-generate(before processing). - Immediately returns acknowledgement as required but doesn't update the ResponseErrorCode and ResponseHttpStatus code in case the message is filtered .
    2) Source queue (OFF) - Response - Auto-generate(after source transformer) - This seemed to be the most close to what I want. But in this case eventhough it actually shows the correct respnseStatus returned by the filter but the response is not immediate . It waits for the Destination to complete the processing before responding. This causes delay on the source side to receive acknowlegement in case there is a retry logic gong on in the destination.

    I also tried with Respnse - Auto-generat( after Destination Complete) - Destination1 - postprocessor but all of them waits for the destination to complete processing before returning the acknowledgement.

    Is there a way to setup such that the acknowledgement is returned immediately after the message is processed on the source transformer side thereby returning any filtering errors that happen ?


    Attached Files

  • #2
    When you have the Source Queue (OFF) it doesn't matter which response option you select, the response will always be sent after processing is complete.

    A destination is considered finished once it returns a status, which can be QUEUED, so if you tell your destination to always queue that can shorten the time to send a response upstream. The destination queue then processes asynchronously in a separate thread from the source thread, which can start accepting the next message.

    Comment


    • #3
      Thanks for the suggestion. That definitely makes sense . But when I check my channel ( as can be seen in the attached screenshot) I do have queue enabled to 'Always' . Still it waits for the destination to complete processing. Is it because I am using ChannelWriter as the destination type and using javascript code called from the destination transformer to send the message to the endPoint ?
      Attached Files

      Comment


      • #4
        Hmm, that shouldn't matter. After the source transformer finishes, it should create the raw content for the destination, and the destination should go directly to queued status before it tries to run the destination transformer. Then the post processor should fire (if there is one,) and then the response should be sent back upstream. It shouldn't need to wait for the destination queue to pick up the message and start processing, regardless of the type of destination connector.

        Can you share a screenshot of all of the received and response timestamps on your message from the message browser?

        Comment


        • #5
          Hey sorry for the delay . Please find as attached 2 screenshots :
          1) My source queue setup - currently set to OFF and Auto-generate(After Source trasformer)
          2) Screenshot of the mesage browser . You can see the first message has a delay of about 16 seconds . All the prior messages were immediate as the SourceQueue was set up as ON.

          Please let me know if you need more information.

          Thanks
          Attached Files

          Comment


          • #6
            Can you try putting a long sleep in your destination transformer? This will sleep it for a minute.
            Code:
            java.lang.Thread.sleep(60000)
            That should be long enough to tell if your source response is really waiting on the destination or not. Your source transformer is running really fast, because you can see the destination received is only 15ms after the source.

            I also noticed that you do have something in your post-processor script in your channel you attached to your original post. The post-processor will need to run before the source sends a response, so long running code there can be a potential problem, too. If your destination is going straight to queue, the post-processor should kick off right after that happens without waiting for the destination to run.

            Comment


            • #7
              I think I have this same use case, my set up is as follows on an http listener:
              • Source Queue: Off
              • Response: PostProcessor that returns $c('responseContent')
              • Filtering - actually "filtering" occurs in the source transformer so I can check stuff, but a good message does go to the destination.
              • ACK is generated in Source Transformer and set to the $c('responseContent')
              • Then the destination is JS that routes to other channels - with those other channel ACK'ing right away (or not)
              This is very fast. My particular use case is one message type can be asynchronous (i.e. send to other channel and we don't care about the ultimate result) but others need to be real time.

              Diridium Technologies, Inc.
              https://diridium.com

              Comment


              • #8
                You can put $r('responseContent') in the responseMap from your source transformer and skip the post-processor step.

                Comment


                • #9
                  True. My feeble mind likes the concept of knowing the response was set in $c. Not directly.
                  Diridium Technologies, Inc.
                  https://diridium.com

                  Comment


                  • #10
                    I believe it still shows up in the mappings area of the message viewer in the same way as the channelMap.

                    Comment


                    • #11
                      Thanks for you inputs . I tried adding a 1 minute delay and also removing all the code from postProcessor . But as visible in the screenshot it still shows one minute delay between Receive and Response times on source side . I have attached the messagebrowser screenshot as well as the updated channel .
                      Attached Files

                      Comment


                      • #12
                        Send the message to another channel to do the work that you have in the channel writer at the moment and have THAT channel respond right away (i.e. have it ACK right away).

                        So into your API, API sends to other channel via router.routeMessage - that other channel ACKs right away and does the real (and asynchronous work)

                        And thanks for good questions and sharing enough info to help as usual!
                        Last edited by pacmano; 04-01-2021, 09:18 AM.
                        Diridium Technologies, Inc.
                        https://diridium.com

                        Comment


                        • #13
                          You mean another channel or a chained destination to the same channel ?

                          Comment


                          • #14
                            Another channel

                            Client -> (API Listener -> API Destination) sends to other channel that has source queue on and Auto Generates Response before processing.

                            This way your listener thinks the message has been processed.
                            Last edited by pacmano; 04-01-2021, 09:30 AM.
                            Diridium Technologies, Inc.
                            https://diridium.com

                            Comment


                            • #15
                              That is cool ! It worked thanks . But shouldn't this behavior be out of the box ideally when Sender queue is set to OFF and 'after source transformer processing' ?

                              Comment

                              Working...
                              X