Announcement

Collapse
No announcement yet.

Best Practices for Pruning Messages

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Best Practices for Pruning Messages

    Hello All,

    I appear to be having an issue with Mirth. Every day at midnight, Mirth halts for ~1 hour as it prunes out old messages. This causes significant problems for our ER and Radiology departments as they have to wait over an hour for 1 study to come across.

    My question is: Are there best practices for pruning to minimize this?

    Currently pruning is set at the defaults of starting at midnight and pruning messages older than 30 days. Last night's batch pruned ~39,000 messages. Is there a way to split this out or another setting to change that will make this smoother? Mirth version is 2.1.1.5490. Thanks.

  • #2
    We try to do it at the least active time.

    Are you doing backups? The default time for that is also midnight. So, you could be doing backups, pruning and usual business @ the same time.

    If you wanted to batch it, you could create a channel that deletes old messages that kicks of every X amount of time.

    Comment


    • #3
      If backing it up is part of a default configuration in Mirth, then probably. I have never seen anywhere to backup on a schedule, though. Where would that be?

      These messages are currently set to prune after 30 days. Batch Pruning is enabled and the Pruning block size is "0". These settings were already in there.

      Comment


      • #4
        Can you set individual times for pruning for each channel? I see that you can set the number of days to retain for each channel, but not times. Would that slow it down or speed it up?

        Comment


        • #5
          You can't do it by channel but you can change the time. Under 'Mirth Connect' select 'Settings' then the 'Message Pruner' tab. Changing the days of pruning would make it slightly faster. You are pruning 39,000 messages set at 30 days. That would be and averate of 1300 messages per day that are coming in. If you changed the pruning days to 25, you would be only 32,500 messages. Your best bet would be to change the time that the pruning occurs.

          Comment


          • #6
            Ok, so another question. Does pruning take roughly the same amount of time regardless of whether you do it daily, weekly, or monthly? The reason i ask is because i have this currently set to daily at midnight but the end time is almost exactly the same every day and sometimes messages pruned will be 40K and sometimes the messages pruned will be 10K. So the time for it to run appears to be the same regardless of the amount of messages actually pruned.

            Comment


            • #7
              Usually the difference in time to delete a 10K - 40K SQL records is relatively insignifigant. Because Mirth stores the message 3 times for each record (raw, transformed and encoded) the records tend to be large. I have not done any benchmarks with Mirth to see how that effects the delete.

              I believe the majority of the time is spent rebuilding indexes. Which depends on how the re-indexing is done. If just changes the indexes for deleted records, there will be a difference based on number of records. If it does a complete re-index, it won't make a difference.
              Last edited by cory_cole; 07-18-2013, 10:54 AM.

              Comment


              • #8
                Ok, thanks. My guess based upon what you said below is that it does a complete re-index because the times are so close whether it is 10K or 40K messages.

                Comment


                • #9
                  Which database are you using (Postgres, SQL Server, etc.)?

                  Comment


                  • #10
                    I don't see a SQL Server instance so i am guess Postgres? I believe I just took the defaults but it has been a while since this was originally set up. Is there a way to see for sure or are those the only two options?

                    Comment


                    • #11
                      It is probably Postgres but I am not sure where you would look to find that out. We have a contract with Mirth and we have the Mirth Appliance. So we do alot of maintenance from there. We have had issues with PostGres 8 not vaccuming correctly which has caused performance issues (all-round not just on pruning). You may want to try re-indexing the message table.

                      http://www.postgresql.org/docs/8.3/s...l-reindex.html

                      Comment


                      • #12
                        OK, thanks. Might just pop up a new mirth station and use SQL Server. As of now, I am setting it to 3 AM as that is their slowest time. I might run some tests in the future to see if setting it to prune weekly helps as well. Also might try putting it on faster hardware or splitting the channels across multiple workstations.

                        Comment


                        • #13
                          Enable Batch Pruning

                          Originally posted by wit-man View Post
                          Hello All,


                          Currently pruning is set at the defaults of starting at midnight and pruning messages older than 30 days. Last night's batch pruned ~39,000 messages. Is there a way to split this out or another setting to change that will make this smoother? Mirth version is 2.1.1.5490. Thanks.
                          Enable Batch Pruning and set Block size to 10000.

                          100,000+ messages are being deleted nightly. Without the batch the Mirth server comes to a halt. I've never tried increasing the number as it worked well with this value. MS SQL Server is being used.

                          Comment


                          • #14
                            Does this allow messages to still go through while it is doing this? Does it run the batches one after the other? Meaning, if I have 40K messages, it will do 10K, then right after that do another 10K, or does it wait for a period of time before doing the next batch?

                            Comment


                            • #15
                              I should process messages in between because they would be in queue for the next lock.

                              Comment

                              Working...
                              X