Announcement

Collapse
No announcement yet.

Automatic channel-centric logging

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Automatic channel-centric logging

    This approach is outdated. Use the much more powerful method described in this thread!

    Functionality:
    The following function(s) provide automatic logging to separate log files for each channel. If a new channel is created, a new log file will automatically be added.

    How to use:
    1. Simply add the code below as code template. (or use the attached code template)
    2. Please assure that the scope global scripts is not activated for this template
    3. If you used the overwriteCategories()-Template remove it (it's functionality is included)
    4. Like overwriteCategories(), a call to activateChannelLogging() has to be added to inbound and outbound transformers before the 1st transformer. For all other events it will be called automatically
    5. All log-files will be written to the same location like mirth.log


    Code:
    Code:
    /**
        Extends standard logging by adding the channel name to the log entries and creating a separate log file for each channel.
        The general mirth log will still log all messages from all channels.<br/>
        <br/> 
        <i>Channel-centric logging is round robin with 10 backups and 2MB file size. These parameters can be changed via the <u>configuration map</u> by defining the following parameters:
            <ul>
                <li><<b>logFile_size</b> - defines the logfile size (eg. 500KB, 3MB, 1GB; default: <b>2MB</b>)</li>
                <li><<b>logFile_backupFiles</b> - defines the number of backup files (e.g. 5; default: <b>10</b>)</li>
                <li><<b>logFile_logLevel</b> - defines the log level (e.g. TRACE, INFO, WARN, ERROR; default: <b>DEBUG</b>)</li>
            </ul>
        </i>
    */
    function activateChannelLogging() {
        if(!channelName){
            channelName = ChannelUtil.getDeployedChannelName(channelId);
        }
        if(logger.getName() == 'undefined'){
            return;    
        }
        // only modify the logger once
        if(!logger.getAppender(channelName)){ 
            
            logger = org.apache.log4j.Logger.getLogger({
                'transformer': channelName + '-transformer',
                'preprocessor': channelName + '-preprocessor',
                'postprocessor': channelName + '-postprocessor',
                'deploy': channelName + '-deploy',
                'shutdown': channelName + '-shutdown',
                'filter': channelName + '-filter',
                'db-connector': channelName + '-db-connector',
                'js-connector': channelName + '-js-connector',
                'attachment': channelName + '-attachment',
                'batch': channelName + '-batch',
                'response': channelName + '-response'
            }[logger.getName()]);
            logger.setLevel(org.apache.log4j.Level.toLevel(configurationMap.get('logFile_logLevel'), org.apache.log4j.Level.DEBUG));
            
            //assign the appender to the Logger
            logger.addAppender(getChannelAppender(channelName));
        }
    }
    
    /**
        Provides a rolling file appender for a channel. If it does not yet exist, it will be created.<br/>
        <br/>    
        <i>The following parameters can be changed via the <u>configuration map</u>:
            <ul>
                <li><<b>logFile_size</b> - defines the logfile size (eg. 500KB, 3MB, 1GB; default: <b>2MB</b>)</li>
                <li><<b>logFile_backupFiles</b> - defines the number of backup files (e.g. 5; default: <b>10</b>)</li>
            </ul>
        </i>
        
        @param {String} channel - the name of the channel for which the rolling file appender is needed
        @param {Object} segment - The acc segment
        @param {string} messageDbKey - The value of the MSH message to be referenced
        
          @return The rolling file appender
    */
    function getChannelAppender(channel){
        // if the appender not yet exists, create it
        if(!globalMap.get('logAppender_' + channel)){
            // create a round robin file appender
            var channelAppender = new org.apache.log4j.RollingFileAppender();
            // use the channel name as appender name
            channelAppender.setName(channel);
            // set the max file size
            channelAppender.setMaxFileSize(configurationMap.get('logFile_size') ? configurationMap.get('logFile_size') : "2MB");
            // set the max number of backup files
            channelAppender.setMaxBackupIndex(configurationMap.get('logFile_backupFiles') ? configurationMap.get('logFile_backupFiles') : 10);
            // determine the channelAppender name and location
            channelAppender.setFile(getLoggingPath() + channel + '.log');
            // define the logging pattern: <Date> <Priority> <Category> <log-Message>
            channelAppender.setLayout(new org.apache.log4j.PatternLayout("%d %-5p %c: %m%n"));
            // set the logging threshold (trace entries will be ignored)
            channelAppender.setThreshold(org.apache.log4j.Level.DEBUG);
            // if preexisting new log entries will be appended
            channelAppender.setAppend(true);
            // activate configuration
            channelAppender.activateOptions();
            // store the new appender in the channel map
            globalMap.put('logAppender_' + channel, channelAppender);
        }
        
        return globalMap.get('logAppender_' + channel);
    }
    
    /**
      Determine the logging path that has been defined for log4j
      
      @return The configured logging path
    */
    function getLoggingPath(){
        var logPath = '';
        // get all appenders
        var appenderList =  org.apache.log4j.Logger.getRootLogger().getAllAppenders();
        // iterate over them
        while(appenderList.hasMoreElements()){
            var appender = appenderList.nextElement();
            // and look for a file appender
            if (appender instanceof org.apache.log4j.FileAppender){
                logPath = appender.getFile();
                // file appender was found, extract path
                var index = (logPath.lastIndexOf('/') != -1) ? logPath.lastIndexOf('/') : logPath.lastIndexOf('\\');
                logPath = logPath.substring(0, index + 1);
                break;
            }
        }    
    
        return logPath;
    }
    
    // automatically call the function
    activateChannelLogging();
    How it works:
    • activateChannelLogging()
      This is the only function that should bother you as you have to call it from the in- and outbound transformers. It adds the custom categories once to each logger and assigns a custom logfile
    • getChannelAppender()
      Creates a custom rolling file appender for each channel and assigns it to all loggers that belongs to this channel
    • getLoggingPath()
      This function determines the path that has been configured for mirth logging


    Customization:
    Certain parameters can be customized by adding parameters to the configuration map (Administrator: Settings ==> Configuration Map):
    logFile_size - defines the logfile size (eg. 500KB, 3MB, 1GB; default: 2MB)
    logFile_backupFiles - defines the number of backup files (e.g. 5; default: 10)
    logFile_logLevel - defines the log level (e.g. TRACE, INFO, WARN, ERROR; default: DEBUG)
    Please be aware that changes to these parameters only become active after the mirth service has been restarted as appenders only will be created once (usually at initial deploy) and kept in memory.

    Further improvements are welcome
    Attached Files
    Last edited by odo; 11-13-2019, 05:02 AM.

  • #2
    @mirth staff: It would be great if this functionality could be added to the core as it would be of great help when you have to manage 50+ channels.

    Further, the reason for needing to explicitly call the function within the transformers is the following code in com.mirth.connect.server.builders.JavaScriptBuilde r:
    Code:
    if (!response) {
          builder.append(" phase[0] = 'transformer'; logger = Packages.org.apache.log4j.Logger.getLogger(phase[0]);");
    }
    I don't understand why a special way for initializing the logger (w/o channel-context) is used here.
    Is it simply a code leftover from the past? Removing the bold part should eliminate the need for adding explicit calls to activateChannelLogging() from within the transformers.

    Comment


    • #3
      Looks awesome! We do have an issue for this: MIRTH-3269

      The reason that logger is re-initialized in the transformer is because the filter and transformer are run as one script for efficiency.
      Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

      Nicholas Rupley
      Work: 949-237-6069
      Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


      - How do I foo?
      - You just bar.

      Comment


      • #4
        I have added activateChannelLogging() to the top of my source and destination transforms, but I am not seeing any logging to my channel's file name.

        I did see a new file show up:
        Code:
        /opt/mirthconnect/logs/<channelName>.log
        I have the Configuration Map set up:
        http://imgur.com/HaI3xhW

        I have the call added to my source transform:
        http://imgur.com/mprjIfS

        Adding the second line to my source transform explicitly calling the logger only ends up putting the TRACE call into the main log file and not my per-channel log file.

        I did also restart my Mirth service before the test, and you can see this output from the /opt/mirthconnect/logs/mirth.log rotating file:
        Code:
        INFO  2017-07-29 00:21:34,187 [Shutdown Hook Thread] com.mirth.connect.server.Mirth: shutting down mirth due to normal request
        INFO  2017-07-29 00:21:44,333 [Main Server Thread] com.mirth.connect.server.Mirth: Mirth Connect 3.4.1.8057 (Built on June 2, 2016) server successfully started.
        INFO  2017-07-29 00:21:44,335 [Main Server Thread] com.mirth.connect.server.Mirth: This product was developed by Mirth Corporation (http://www.mirthcorp.com) and its contributors (c)2005-2017.
        INFO  2017-07-29 00:21:44,335 [Main Server Thread] com.mirth.connect.server.Mirth: Running Java HotSpot(TM) 64-Bit Server VM 1.8.0_101 on Linux (3.13.0-117-generic, amd64), postgres, with charset US-ASCII.
        INFO  2017-07-29 00:21:44,336 [Main Server Thread] com.mirth.connect.server.Mirth: Web server running at http://172.17.0.15:8080/ and https://172.17.0.15:8443/
        TRACE 2017-07-29 00:22:24,879 [Source Filter/Transformer JavaScript Task on Lexington-to-Curatess inMDM TEST (0aa3d9d0-ec7f-40f9-97fa-d6a5af53493d) < pool-6-thread-1] Lexington-to-Curatess inMDM TEST-transformer: START source transform
        TRACE 2017-07-29 00:22:24,884 [Source Filter/Transformer JavaScript Task on Lexington-to-Curatess inMDM TEST (0aa3d9d0-ec7f-40f9-97fa-d6a5af53493d) < pool-6-thread-1] Lexington-to-Curatess inMDM TEST-transformer: END source transform
        Last edited by Eric_R; 07-28-2017, 04:54 PM.
        Eric Richards | Datica | Madison, WI

        Comment


        • #5
          Very cool. I've been making sure that my channels always use the channel name in their logging code so it helps me to understand what's going on with which channel in the console of mirth.

          Your solution must have come of similar needs. I'm going to see about implementing this, my channel count is over just over 50, so I'm sure to see some useful assistance!

          thanks for your work on this, I'm sure it was a time-consuming event to get everything just so. Once implemented, I'll get you some feedback, positive or negative...

          thanks,
          Jack

          Comment


          • #6
            Originally posted by Eric_R View Post
            Adding the second line to my source transform explicitly calling the logger only ends up putting the TRACE call into the main log file and not my per-channel log file.
            Currently, the appender is ignoring the trace level. Try to log something to level debug or above. It should appear in the log file of the channel.

            For also logging at trace level, you have to modify the function getChannelAppender()

            The following line:
            Code:
            // set the logging threshold (trace entries will be ignored)
            channelAppender.setThreshold(org.apache.log4j.Level.DEBUG);
            has to be changed to:
            Code:
            		
            // set the logging threshold
            channelAppender.setThreshold(org.apache.log4j.Level.TRACE);

            I actually don't remember why I added this threshold. (as there does not seem to be a default value, it might also work fine w/o this line)

            Please let me know if this works.
            Last edited by odo; 08-08-2017, 04:44 AM.

            Comment


            • #7
              Originally posted by jack.downes View Post
              Your solution must have come of similar needs. I'm going to see about implementing this, my channel count is over just over 50, so I'm sure to see some useful assistance!
              Yes, we are using it for more than 400 channels on 5 different servers (prod & dev) and it works very reliable for our needs.

              Only limitation so far is that exceptions are only logged to the main log file (mirth.log). It would need a minor adjustment to the mirth connect code to allow automated exception logging (and also automated transformer logging w/o explicitly calling activateChannelLogging() first).

              Comment

              Working...
              X