Announcement

Collapse
No announcement yet.

Queueing due to reprocessing large number of messages

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Queueing due to reprocessing large number of messages

    We have the situation where we are reprocessing 2.6 million messages in a channel. The channel has source queue ON. The request to reprocess that many messages take quite a while to retrieve them from the db and put them back into the channel. The source queue is ON and everything works as expected.

    In the meantime, this channel is also receiving live incoming new data from an external source, as normal real-time message processing.

    My question is, for these live feed incoming messages, when they hit the channel that is in the process of queueing the messages for the reprocessing, what happens to these new live feed incoming messages? When then come into this channel that is busy queueing the reprocess, do these messages get intermixed into the queue, or do they wait somewhere for the channel to finish queueing the reprocessed messages?

    I'm concerned about message order, and that maybe we need to be pausing the live stream until the reprocessing messages all get queued?

    Thanks for you time!
    -kevin

  • #2
    Do you see a backload of messages in the source queue?
    Best,

    Kirby

    Mirth Certified|Epic Bridges Certified|Cloverleaf Level 2 Certified

    Appliance Version 3.11.4
    Mirth Connect Version 3.8.0
    Java Version 1.6.0_45-b06
    Java (64 bit) Version 1.6.0_45-b06
    Java 7 (64 bit) Version 1.7.0_151-b15
    Java 8 (64 bit) Version 1.8.0_181-b13
    PostgreSQL Version 9.6.8

    Comment


    • #3
      Yes, the source queue is queueing as expected, there are many messages (several days of processing worth) actively being added into the source queue. In the meantime we have not turned off our live feed. So, my question is where do the new incoming live feed messages go? At the end of the queue, after all the reprocessing messages finally get queued? Or do the new incoming live feed messages get inserted into the queue as they come in, which would be before all the reprocessing messages get fully queued.
      Last edited by khobbs; 03-09-2020, 01:11 PM.

      Comment


      • #4
        If you are reprocessing as new messages, I'm almost certain that the live messages will be interspersed with reprocessed messages as they are being created.

        I'm not sure how it will affect processing order if you are replacing the old messages.

        Comment

        Working...
        X