Announcement

Collapse
No announcement yet.

New Feature: Mirth logging replacement

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • New Feature: Mirth logging replacement

    Functionality
    This library eliminates Mirth logging limitations by replacing the logging mechanism.
    • All log messages created by a channel will be placed in a separate log file named like the channel itself.
    • Messages that are not caused by a channel (e.g. global deploy script) are logged to the standard log file mirth.log.
    • All messages on log level ERROR are accumulated in a specific log file called mirthErrors.log.
    • All messages that are logged to the console or administrator dashboard are enriched with the name of the channel causing the message.

    How to use:
    1. Copy the attached library MetaAppender.jar to the custom-lib subfolder of your mirth installation
    2. Add the following code to the global deploy script:
      Code:
      Packages.lu.hrs.mirth.MetaAppender.activate();
      (just needs to be called once after the service is started but multiple calls do not do harm)
    3. If you are using the function activateChannelLogging(), remove it or remove it's content if already referenced in many places
    4. restart the mirth service


    Customization:
    By default the logging configuration of log4j.properties in the subfolder .\config is used. It is however possible to overwrite certain parameters by providing them to the activate() call:

    Packages.lu.hrs.mirth.MetaAppender.activate(<customLogPath>, <customMaxFileSize>, <customMaxBackupIndex>, <customLogPattern>, <logAllToMainLog>);
    customLogPath - Defines a custom location for the log files
    customMaxFileSize - Defines a custom maximal size per log file
    customMaxBackupIndex - Defines a maximum number of log files that will be created per channel till the oldest is overwritten (round-robin)
    customLogPattern - Defines a custom structure for the log file entries
    logAllToMainLog - If this flag is set, all log entries (also channel-specific ones) will also be logged to the main log. (off by default)

    All parameter are optional and can be expressed by null. Tailing parameters can be omitted.

    Examples:
    Writes the channel-specific log entries in a custom structure:
    Code:
    Packages.lu.hrs.mirth.MetaAppender.activate(null, null, null, "%d %-5p %c: %m%n");
    Extends the size of all log files to 5MB and logs all channel data also to the main log file (mirth.log):
    Code:
    Packages.lu.hrs.mirth.MetaAppender.activate(null, '5MB', null, null, true);
    New Features:
    Focus on specific channel log
    If many channels are logging to the dashboard, you might want to focus on the log output of one specific channel if e.g. an issue occurs. This can be done by:
    Code:
    Packages.lu.hrs.mirth.MetaAppender.setFocus(<Channel name or id>);
    Focus can be removed by:
    Code:
    Packages.lu.hrs.mirth.MetaAppender.removeFocus();
    • Setting a focus does not influence the logging to the channel log-files.
    • If a focus is set, this is indicated by a "FOCUSED: "-prefix before the channel name of each log entry.
    Suppress specific channel log
    In case of an issue with a specific channel, you might want to log additional information for tracking down the bug (maybe even in prod env ) . However, while having additional information in the channel log, you probably do not want to have your dashboard log spammed.
    This can be done via:
    Code:
    Packages.lu.hrs.mirth.MetaAppender.setFilter(<Channel name or id>);
    Focus can be removed by:
    Code:
    Packages.lu.hrs.mirth.MetaAppender.removeFilter();
    • Setting a filter does not influence the logging to the channel log-files but only the dashboard.
    • If a filter is set, this is indicated by a "FILTERED: "-prefix before the channel name of each log entry.


    Additional Information:
    • The meta appender "hijacks" the mirth logging mechanism and gains that way full control over all logging activities.
    • It thus substitutes the approach I posted some time ago. It overcomes all weaknesses of this initial approach (no exception logging to channel logs, no automated integration for transformers).
    • It solves all issues mentioned here in Mirth Jira
    • Source code is included in the jar file

    This project can now be found on GitHub
    Attached Files
    Last edited by odo; 05-26-2020, 11:15 PM. Reason: Updated library to latest version

  • #2
    Thats awesome!

    Since it extends the Log4J appender, does it automatically rotate logs based on log4j.properties?

    What license is this code under?
    Jon Bartels

    Zen is hiring!!!!
    http://consultzen.com/careers/
    Talented healthcare IT professionals wanted. Engineers to sales to management.
    Good benefits, great working environment, genuinely interesting work.

    Comment


    • #3
      Originally posted by jbartels View Post
      Thats awesome!
      Thanx


      Originally posted by jbartels View Post
      Since it extends the Log4J appender, does it automatically rotate logs based on log4j.properties?
      Yes. It uses the configuration in .\conf\log4j.properties and .\conf\log4j-cli.properties as master for all logs (also the channel-specific ones).

      It can however be customized at runtime by providing the parameters described in the first post to the activate() call.


      Originally posted by jbartels View Post
      What license is this code under?
      MPL-1.1 (same as Mirth)

      Comment


      • #4
        Added "focus"-functionality to temporarily limit dashboard logging to a specific channel.

        View first post for details.

        Comment


        • #5
          Works like a charm! Really like it.
          I did have a problem getting it to run, don't know why. Got a null reference exception but no further info (was wrapped in the deploy script trace).
          Could not get it working until i changed one of the log levels to "WARN", i had them all on "ERROR".

          Comment


          • #6
            Originally posted by V3nn3tj3 View Post
            Works like a charm! Really like it.
            I did have a problem getting it to run, don't know why. Got a null reference exception but no further info (was wrapped in the deploy script trace).
            Could not get it working until i changed one of the log levels to "WARN", i had them all on "ERROR".
            That's strange as it should work out of the box w/o the need of any modification of the configuration file. But glad to read you got it working!

            Unfortunately I have not been able to reproduce this exception. What are your specs (mirth version, OS, Java version)? Are you able to reproduce the Exception with more details?

            Comment


            • #7
              It's awesome that is a way to extend the functionality of the Log4j Appender.

              I've tryed it at my mirth, but i getting an Transformer error when i call the logger function.
              Do you have any idea to fix this?
              I'm running a Mirth in the version 3.5.2.

              Here's the stacktrace of the exception:

              Transformer error
              ERROR MESSAGE: Error evaluating transformer
              com.mirth.connect.server.MirthJavascriptTransforme rException:
              CHANNEL: KOKKM_GenericMDM
              CONNECTOR: sourceConnector
              SCRIPT SOURCE:
              SOURCE CODE:
              725: var appender = appenderList.nextElement();
              726: // and look for a file appender
              727: if (appender instanceof org.apache.log4j.FileAppender){
              728: logPath = appender.getFile();
              729: // file appender was found, extract path
              730: var index = (logPath.lastIndexOf('/') != -1) ? logPath.lastIndexOf('/') : logPath.lastIndexOf('\\');
              731: logPath = logPath.substring(0, index + 1);
              732: break;
              733: }
              734: }
              LINE NUMBER: 730
              DETAILS: TypeError: Cannot call method "lastIndexOf" of null
              at f9f09143-96ce-4fb4-8087-54e59eb77428:730 (getLoggingPath)
              at f9f09143-96ce-4fb4-8087-54e59eb77428:698 (getChannelAppender)
              at f9f09143-96ce-4fb4-8087-54e59eb77428:666 (activateChannelLogging)
              at f9f09143-96ce-4fb4-8087-54e59eb77428:740
              at com.mirth.connect.server.transformers.JavaScriptFi lterTransformer$FilterTransformerTask.doCall(JavaS criptFilterTransformer.java:154)
              at com.mirth.connect.server.transformers.JavaScriptFi lterTransformer$FilterTransformerTask.doCall(JavaS criptFilterTransformer.java:119)
              at com.mirth.connect.server.util.javascript.JavaScrip tTask.call(JavaScriptTask.java:113)
              at java.util.concurrent.FutureTask.run(Unknown Source)
              at java.util.concurrent.ThreadPoolExecutor.runWorker( Unknown Source)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run (Unknown Source)
              at java.lang.Thread.run(Unknown Source)

              Comment


              • #8
                The reason is that you are using my old approach - activateChannelLogging() - in parallel.

                This is neither necessary nor recommended. Please remove the activateChannelLogging() function or, if you already referenced it in many transformers, simply remove the content of the function. I will emphasize this in the first post.

                There is no need anymore to reference a function in the transformer or anywhere else (well besides once in the global deploy script) for activating advanced logging.

                (I also added further functionality to the jar for focusing and filtering of multiple channel logs on the dashboard. It will be released as soon as I get some time)
                Last edited by odo; 01-23-2020, 02:36 PM.

                Comment


                • #9
                  Yes, you're right. I had the activateChannelLogging() in several transformer.
                  Thanks! Now it's working.

                  Comment


                  • #10
                    Not able to redirect log file path

                    Thank you for developing this useful tool!

                    I am trying to use it to make accessing Mirth's logs easier so that I can use them in unit tests of my channels.

                    I am running Mirth 3.8.1 on Ubuntu 18.04.3 LTS.

                    I have installed the plugin and activated it in my global deploy script - I can see the channel-specific logs now being generated in /opt/mirthconnect/logs.

                    However, I am not able to get the customLogPath feature to work. I have tried quite a few different variations in my activation call, providing paths with a file name and without, in my home directory and in subfolders of Mirth's default logging folder:

                    Packages.lu.hrs.mirth.MetaAppender.activate("/opt/mirthconnect/logs/test/test.log");

                    However, the logs always just go to the default folder and default channel-specific log files.

                    Any advice would be much appreciated.

                    Comment


                    • #11
                      Originally posted by hheldenbrand View Post
                      However, I am not able to get the customLogPath feature to work. I have tried quite a few different variations in my activation call, providing paths with a file name and without, in my home directory and in subfolders of Mirth's default logging folder:

                      Packages.lu.hrs.mirth.MetaAppender.activate("/opt/mirthconnect/logs/test/test.log");

                      In the current version, the meta appender is only created once, at the first call (usually the 1st time the global deploy script is called).
                      All subsequent calls will returned the cached instance. This is done for performance reasons.


                      Thus, if you would like a custom log path, you would have to provide this parameter where activate is called the first time. Alternatively you can simply change the log path in the configuration file (<MIRTH HOME>\conf\log4j.properties)

                      This custom path is however used for all log files. The logfile names are fix. (e.g. log files of channels are named like the channel). Thus, please provide the logpath without a filename.


                      I might change the code in a future version to detect, when parameters are changed. This, however, comprises a potential risk as two alternating calls from different locations with different values e.g. for the logfile location might result in unwanted behavior.


                      EDIT: When I think about it: An additional feature similar to the focus() feature might be a feasible way for securely supporting custom locations for single log files. I will have a look at that as soon as I get some time.
                      Last edited by odo; 01-29-2020, 06:46 AM.

                      Comment


                      • #12
                        odo, have you considered uploading your source code to github?

                        Comment


                        • #13
                          Originally posted by agermano View Post
                          odo, have you considered uploading your source code to github?
                          Project is now available on GitHub

                          Comment


                          • #14
                            Awesome!

                            Comment


                            • #15
                              I must be missing something here?

                              I have added the line
                              Code:
                              Packages.lu.hrs.mirth.MetaAppender.activate();
                              To my Global scripts. It does not mention if the drop down should be one of the following Deploy, Preprocessor or post processor.
                              I tired all 3 in this case.

                              I was expecting a new log file for each channel i have. Since this is in my test lab i have setup 5 channels.
                              My mirth config is set to use a SQL database and not the local derby database.
                              I have tried chnaging the log level for the 3 diffrent log types i see in the Connect Server Manager

                              I am sure i am missing something here just i cant figure out what
                              Am i wrong to think that i should get a log per channel i have?
                              I was hopeing to see info level details for the channels but i see nothing yet.
                              I do get a new channel log if i get the error below when i try to set a value in the
                              Code:
                              Packages.lu.hrs.mirth.MetaAppender.activate();
                              within my global script.

                              I am sure i am doing something wrong so any help is very much apreciated.


                              if i use this

                              Packages.lu.hrs.mirth.MetaAppender.setFocus(<Channel name or id>);
                              i get nothing new in logging.

                              The only time i get an error is when i try to add valuses to the activate
                              Code:
                              Packages.lu.hrs.mirth.MetaAppender.activate(null, '5MB', null, null, true);
                              I get the following error in the console
                              Code:
                              Can't find method lu.hrs.mirth.MetaAppender.activate(null,string,null,null,boolean)
                              This is the full error
                              Code:
                              [2020-05-26 12:53:17,244] ERROR (com.mirth.connect.server.controllers.DonkeyEngine Controller:1530): Error executing global deploy script.
                              com.mirth.connect.server.util.javascript.JavaScrip tExecutorException: com.mirth.connect.server.MirthJavascriptTransforme rException: SOURCE CODE:
                              52: }53: eval('importPackage(' + Packages.java.lang.Class.forName(className).getPac kage().getName() + ')');54: }55: }56: function doScript() {57: Packages.lu.hrs.mirth.MetaAppender.activate(null, '50MB', null, null, true);58: return; 59: }LINE NUMBER:
                              57DETAILS:
                              Can't find method lu.hrs.mirth.MetaAppender.activate(null,string,nul l,null,boolean).
                              at Deploy:57 (doScript)
                              at Deploy:60
                              at com.mirth.connect.server.util.javascript.JavaScrip tUtil.execute(JavaScriptUtil.java:75)
                              at com.mirth.connect.server.util.javascript.JavaScrip tUtil.executeGlobalDeployScript(JavaScriptUtil.jav a:431)
                              at com.mirth.connect.server.controllers.DefaultScript Controller.executeGlobalDeployScript(DefaultScript Controller.java:144)
                              at com.mirth.connect.server.controllers.DonkeyEngineC ontroller.executeGlobalDeployScript(DonkeyEngineCo ntroller.java:1528)
                              at com.mirth.connect.server.controllers.DonkeyEngineC ontroller.deployChannels(DonkeyEngineController.ja va:335)
                              at com.mirth.connect.server.controllers.DonkeyEngineC ontroller.redeployAllChannels(DonkeyEngineControll er.java:460)
                              at com.mirth.connect.server.api.servlets.EngineServle t.redeployAllChannels(EngineServlet.java:46)
                              at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
                              at sun.reflect.NativeMethodAccessorImpl.invoke(Unknow n Source)
                              at sun.reflect.DelegatingMethodAccessorImpl.invoke(Un known Source)
                              at java.lang.reflect.Method.invoke(Unknown Source)
                              at com.mirth.connect.server.api.providers.MirthResour ceInvocationHandlerProvider$1.invoke(MirthResource InvocationHandlerProvider.java:219)
                              at org.glassfish.jersey.server.model.internal.Abstrac tJavaResourceMethodDispatcher$1.run(AbstractJavaRe sourceMethodDispatcher.java:144)
                              at org.glassfish.jersey.server.model.internal.Abstrac tJavaResourceMethodDispatcher.invoke(AbstractJavaR esourceMethodDispatcher.java:161)
                              at org.glassfish.jersey.server.model.internal.JavaRes ourceMethodDispatcherProvider$VoidOutInvoker.doDis patch(JavaResourceMethodDispatcherProvider.java:14 3)
                              at org.glassfish.jersey.server.model.internal.Abstrac tJavaResourceMethodDispatcher.dispatch(AbstractJav aResourceMethodDispatcher.java:99)
                              at org.glassfish.jersey.server.model.ResourceMethodIn voker.invoke(ResourceMethodInvoker.java:389)
                              at org.glassfish.jersey.server.model.ResourceMethodIn voker.apply(ResourceMethodInvoker.java:347)
                              at org.glassfish.jersey.server.model.ResourceMethodIn voker.apply(ResourceMethodInvoker.java:102)
                              at org.glassfish.jersey.server.ServerRuntime$2.run(Se rverRuntime.java:326)
                              at org.glassfish.jersey.internal.Errors$1.call(Errors .java:271)
                              at org.glassfish.jersey.internal.Errors$1.call(Errors .java:267)
                              at org.glassfish.jersey.internal.Errors.process(Error s.java:315)
                              at org.glassfish.jersey.internal.Errors.process(Error s.java:297)
                              at org.glassfish.jersey.internal.Errors.process(Error s.java:267)
                              at org.glassfish.jersey.process.internal.RequestScope .runInScope(RequestScope.java:317)
                              at org.glassfish.jersey.server.ServerRuntime.process( ServerRuntime.java:305)
                              at org.glassfish.jersey.server.ApplicationHandler.han dle(ApplicationHandler.java:1154)
                              at org.glassfish.jersey.servlet.WebComponent.serviceI mpl(WebComponent.java:471)
                              at org.glassfish.jersey.servlet.WebComponent.service( WebComponent.java:425)
                              at org.glassfish.jersey.servlet.ServletContainer.serv ice(ServletContainer.java:383)
                              at org.glassfish.jersey.servlet.ServletContainer.serv ice(ServletContainer.java:336)
                              at org.glassfish.jersey.servlet.ServletContainer.serv ice(ServletContainer.java:223)
                              at org.eclipse.jetty.servlet.ServletHolder.handle(Ser vletHolder.java:845)
                              at org.eclipse.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1712)
                              at com.mirth.connect.server.MethodFilter.doFilter(Met hodFilter.java:37)
                              at org.eclipse.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1699)
                              at com.mirth.connect.server.api.providers.Clickjackin gFilter.doFilter(ClickjackingFilter.java:44)
                              at org.eclipse.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1699)
                              at com.mirth.connect.server.api.providers.ApiOriginFi lter.doFilter(ApiOriginFilter.java:71)
                              at org.eclipse.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1699)
                              at org.eclipse.jetty.servlet.ServletHandler.doHandle( ServletHandler.java:582)
                              at org.eclipse.jetty.server.session.SessionHandler.do Handle(SessionHandler.java:224)
                              at org.eclipse.jetty.server.handler.ContextHandler.do Handle(ContextHandler.java:1180)
                              at org.eclipse.jetty.servlet.ServletHandler.doScope(S ervletHandler.java:512)
                              at org.eclipse.jetty.server.session.SessionHandler.do Scope(SessionHandler.java:185)
                              at org.eclipse.jetty.server.handler.ContextHandler.do Scope(ContextHandler.java:1112)
                              at org.eclipse.jetty.server.handler.ScopedHandler.han dle(ScopedHandler.java:141)
                              at org.eclipse.jetty.server.handler.HandlerList.handl e(HandlerList.java:52)
                              at org.eclipse.jetty.server.handler.HandlerWrapper.ha ndle(HandlerWrapper.java:134)
                              at org.eclipse.jetty.server.Server.handle(Server.java :534)
                              at org.eclipse.jetty.server.HttpChannel.handle(HttpCh annel.java:320)
                              at org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:251)
                              at org.eclipse.jetty.io.AbstractConnection$ReadCallba ck.succeeded(AbstractConnection.java:273)
                              at org.eclipse.jetty.io.FillInterest.fillable(FillInt erest.java:95)
                              at org.eclipse.jetty.io.ssl.SslConnection.onFillable( SslConnection.java:202)
                              at org.eclipse.jetty.io.AbstractConnection$ReadCallba ck.succeeded(AbstractConnection.java:273)
                              at org.eclipse.jetty.io.FillInterest.fillable(FillInt erest.java:95)
                              at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(S electChannelEndPoint.java:93)
                              at org.eclipse.jetty.util.thread.strategy.ExecuteProd uceConsume.executeProduceConsume(ExecuteProduceCon sume.java:303)
                              at org.eclipse.jetty.util.thread.strategy.ExecuteProd uceConsume.produceConsume(ExecuteProduceConsume.ja va:148)
                              at org.eclipse.jetty.util.thread.strategy.ExecuteProd uceConsume.run(ExecuteProduceConsume.java:136)
                              at org.eclipse.jetty.util.thread.QueuedThreadPool.run Job(QueuedThreadPool.java:671)
                              at org.eclipse.jetty.util.thread.QueuedThreadPool$2.r un(QueuedThreadPool.java:589)
                              at java.lang.Thread.run(Unknown Source)Caused by: com.mirth.connect.server.MirthJavascriptTransforme rException: SOURCE CODE:
                              52: }53: eval('importPackage(' + Packages.java.lang.Class.forName(className).getPac kage().getName() + ')');54: }55: }56: function doScript() {57: Packages.lu.hrs.mirth.MetaAppender.activate(null, '50MB', null, null, true);58: return; 59: }LINE NUMBER:
                              57DETAILS:
                              Can't find method lu.hrs.mirth.MetaAppender.activate(null,string,nul l,null,boolean).
                              at Deploy:57 (doScript)
                              at Deploy:60
                              at com.mirth.connect.server.util.javascript.JavaScrip tUtil.executeScript(JavaScriptUtil.java:547)
                              at com.mirth.connect.server.util.javascript.JavaScrip tUtil$4.doCall(JavaScriptUtil.java:437)
                              at com.mirth.connect.server.util.javascript.JavaScrip tTask.call(JavaScriptTask.java:113)
                              at java.util.concurrent.FutureTask.run(Unknown Source)
                              at java.util.concurrent.ThreadPoolExecutor.runWorker( Unknown Source)
                              at java.util.concurrent.ThreadPoolExecutor$Worker.run (Unknown Source)
                              ... 1 more

                              Comment

                              Working...
                              X