Announcement

Collapse
No announcement yet.

Any ideas on how to monitor queued messages?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Any ideas on how to monitor queued messages?

    Hi,

    I have been having a few problems with some of our client destinations within the last month that have caused various channels to have their messages begin to queue for extended periods of times without anyone noticing.

    I was thinking about creating a channel that would see if any other channel had queued message and then send an email/alert to a few people. I know that I can probably try to query to Postgres database directly, but does anyone know of any Mirth functions/code/javascript/etc that I could use instead?

  • #2
    We have an alerting channel that pulls from the database. It then sleeps for 3 mins and pulls again (to make sure it is not just a data dump). If it is still not decreasing, the channel runs some steps to attempt to fix the issue (stop/start channels, undeploy/redeploy channels). If any of these steps fix the issue, we still get the alert but the email we indicate which step fixed it. If nothing fixes it, the channel will run diagnostics (test connection to the ports, ping the site). The alert will email what steps need to be taken based on the results of the diagnostic.

    Comment


    • #3
      Do you mind sharing that channel?

      Comment


      • #4
        I will try to scale it down by tomorrow. We have things that are specific to us (i.e. naming conventions to prevent alerts on test channels, contact info, etc.)

        Comment


        • #5
          http://www.mirthcorp.com/community/f...ead.php?t=4254

          Comment


          • #6
            Thanks. This seems to fit with what we need. I will have to tinker around with it, but it is doing what it is supposed to do.

            Comment


            • #7
              Here is what I use to send out alerts for messages stuck in queue on specific channels:


              (Put this in the deploy script)

              //This ensures that the alert will trigger if the condition occurs soon after deploying

              var rightnow = new Date();
              var newtime = rightnow - (3 * (1000 * 60 * 60))


              globalMap.put(channelId + '_lastalert',newtime);
              (put this in the postprocessor)

              // This script executes once after a message has been processed

              //Check to see if the alert has already been sent within the given timeframe

              var lastsent = globalMap.get(channelId + '_lastalert'); //last time an alert was sent
              var rightnow = new Date(); //Current Datetime
              var hLimit = 1; //Number of hours before resending an alert

              var datediff = (rightnow - lastsent) / (1000 * 60 * 60); //calculates hours since last alert went out

              if (datediff < hLimit) {

              return;

              } else {

              //This is the bit that does the heavy lifting

              var controller = Packages.com.mirth.connect.server.controllers.Chan nelStatisticsController.getInstance();
              var channelName = Packages.com.mirth.connect.server.controllers.Chan nelController.getInstance().getDeployedChannelById (channelId).getName();
              var stats = controller.getStatistics(channelId);


              //I include all these so you can see what's available
              var mSent = stats.getSent();
              var mReceived = stats.getReceived();
              var mQueued = stats.getQueued();
              var mError = stats.getError();

              var qAlarm = 10;


              if (mQueued % qAlarm == 0 && mQueued != 0) //This checks the limit of queued messages in groups of (qAlarm) 10.

              {

              var emTo = '[email protected]'; //This is the email address you want to send the alert to.
              var emCC = '[email protected]'; //This is any cc emails you want to include
              var emFrom = '[email protected]'; //This is the "From" address that will appear on your email
              var emSubj = 'Queue Alert for Channel (LIVE): ' + channelName; //This is the subject of the email
              var emBody = 'Queued messages detected on channel (LIVE) ' + channelName + '.\n';
              emBody = emBody + 'Current number of queued messages is ' + mQueued.toString() + '.\n';
              emBody = emBody + 'This alert will not be resent for ' + hLimit.toString() + ' hours.\nPlease do not respond to this email. This address is not monitored.';


              var smtpConn = SMTPConnectionFactory.createSMTPConnection(); //Opens email connection using email settings in your Mirth Connect Settings.

              smtpConn.send(emTo,emCC,emFrom,emSubj,emBody);


              globalMap.put(channelId + '_lastalert',rightnow);



              }
              }
              You can change the var qAlarm amount to send the alert for groups of queues. For example, I have 10 in this example and that means that every time 10 new messages get added to the queue an alert is sent out (i.e. 10, 20, 30, etc...).

              Comment


              • #8
                Thank you atibbits, this is priceless!

                Comment


                • #9
                  Cory - Can you share the channel?

                  Comment


                  • #10
                    Can you share the channel?

                    Comment


                    • #11
                      Originally posted by atibbits View Post
                      Here is what I use to send out alerts for messages stuck in queue on specific channels:






                      You can change the var qAlarm amount to send the alert for groups of queues. For example, I have 10 in this example and that means that every time 10 new messages get added to the queue an alert is sent out (i.e. 10, 20, 30, etc...).
                      So, I really liked this concept and Thanks for the code. I took this and modified it for 3.X and made a few changes. The reason for my changes was we use an on-call system and don't monitor emails during the night typically.

                      I placed the Post processor script into Pre-Processor also, because what ended up happening was the queue was on the incoming thread and that was the only thing we weren't really monitoring. This allowed us to see what is being done because the post process wouldn't happen in our issue.

                      I also changed the hour monitor to minutes. This allowed us to change the time frames based on time of day a little easier.

                      Here it is:

                      Deploy:

                      // This script executes once when the channel is deployed
                      // You only have access to the globalMap and globalChannelMap here to persist data

                      var rightnow = new Date();
                      var newtime = rightnow - (3 * (1000 * 60 * 60))


                      globalMap.put(channelId + '_lastalert',newtime);



                      return;



                      Pre-Processor:

                      // Modify the message variable below to pre process data
                      var lastsent = globalMap.get(channelId + '_lastalert'); //last time an alert was sent
                      var rightnow = new Date(); //Current Datetime
                      var hLimit = 0;


                      if (rightnow.getHours() >= 13)
                      {
                      hLimit = 30; //Number of hours before resending an alert
                      var emTo = 'TO Address';
                      var emCC = 'CC Address';
                      }
                      else {

                      hLimit = 60;
                      var emTo = 'TO Address';
                      var emCC = ' ';
                      }

                      var datediff = (rightnow - lastsent) / (1000 * 60); //calculates hours since last alert went out

                      if (datediff < hLimit) {

                      return message;

                      }
                      else {

                      var channelName = ChannelUtil.getDeployedChannelName(channelId);

                      //I include all these so you can see what's available
                      var mSent = ChannelUtil.getSentCount(channelId);
                      var mReceived = ChannelUtil.getReceivedCount(channelId);
                      var mQueued = ChannelUtil.getQueuedCount(channelId);
                      var mError = ChannelUtil.getErrorCount(channelId);

                      var qAlarm = 5;

                      if (mQueued >= qAlarm) //This checks the limit of queued messages in groups of (qAlarm) 10.

                      {
                      var emFrom = 'FROM ADDRESS'; //This is the "From" address that will appear on your email
                      var emSubj = 'Queue Alert for Channel (LIVE): ' + channelName; //This is the subject of the email
                      var emBody = 'Queued messages detected on channel (LIVE) ' + channelName + '.\n';
                      emBody = emBody + 'Current number of queued messages is ' + mQueued.toString() + '.\n';
                      emBody = emBody + 'This alert will not be resent for ' + hLimit.toString() + ' minutes.\nPlease do not respond to this email. This address is not monitored.\n';
                      emBody = emBody + 'Message Sent at ' + rightnow + ' \n';


                      var smtpConn = SMTPConnectionFactory.createSMTPConnection(); //Opens email connection using email settings in your Mirth Connect Settings.

                      smtpConn.send(emTo,emCC,emFrom,emSubj,emBody);

                      globalMap.put(channelId + '_lastalert',rightnow);
                      }
                      }


                      return message;





                      Comment

                      Working...
                      X