Announcement

Collapse
No announcement yet.

Code Template: Automatic Mirth Backup

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Code Template: Automatic Mirth Backup

    Purpose:
    This code template creates a backup of a mirth configuration like if you click "backup config" in the settings tab of the mirth administrator and stores it in the filesystem.

    It backs up the whole mirth configuration and also the configuration map. Everything is compressed into a zip archive and can optionally be secured with 256bit AES encryption.

    We call it from a dedicated channel for generating a daily backup of all production and development mirth instances.

    Parameters:
    • username
      The username that the channel should use to connect to the server that should be backed-up. Dedicated user is recommended.
    • password
      The password that the channel should use to connect to the server that should be backed-up.
    • server
      The ip or name of the mirth server that should be backed-up. It can also contain a custom port (default is 8443). This parameter will become part of the backup name.
    • backupFolder
      Path to the folder where the backup should be created. None-existing parts of the path will be created. Further, all backups of the same day will be placed in a dedicated subfolder.
    • archivePassword
      If a password is set, the resulting archive will be secured by 256bit AES encryption (Optional)


    Examples:
    Create a backup archive without encryption:
    Code:
    backupMirthServer('admin', 'admin', 'MyMirthProdInstance', 'c:\temp');
    Create a 256bit AES encrypted backup archive:
    Code:
    backupMirthServer('admin', 'admin', 'MyMirthProdInstance', 'c:\temp', 'MySecurePassword!');
    Mirth version:
    In production with Mirth 3.4.2/3.6.1/3.7.1, confirmed to be working with Mirth 3.5.1 (but should also work with other versions, however untested - feedback would be nice)

    How to use:
    1. Place the zip4J library in the custom-lib folder of mirth. (It is needed for zip encryption - further info can be found on the website)
    2. Press the button "Reload Resources" under Settings -> Resources in Mirth Administrator
    3. Import the attached code template

    Code:
    /**
        Creates a backup of the whole mirth configuration (like if you press the "backup config" button in Mirth Administrator) 
        and writes it to a folder.<br/>
        <br/>
        <i>The name of each backup is unique & all backups of one day will be written to a dedicated sub-folder named by the date</i>
    
        @param {String} username - The username that the channel should use to connect to the server that should be backed-up.
        @param {String} password - The password that the channel should use to connect to the server that should be backed-up.
        @param {String} server - The ip or name of the mirth server that should be backed-up. It can also contain a custom port. This parameter will become part of the backup name. 
        @param {String} backupFolder - Path to the folder where the backup should be created
        @param {String} archivePassword - If a password is provided, the zip archive will be encrypted with this password
    */
    function backupMirthServer(username, password, server, backupFolder, archivePassword) {
    
        logger.info('Initializing export of mirth server ' + server);
    
        // create a client instance and initialize it with the server to which it should connect
        var client = new com.mirth.connect.client.core.Client('https://' + server + ((server.indexOf(':') == -1) ? ':8443' : ''));
        // create an instance of the serializer used to serialize the configuration to xml
        var  serializer = com.mirth.connect.model.converters.ObjectXMLSerializer.getInstance();
        // log on to the server
        try{
            var loginStatus = client.login(username, password);
        }catch(ex){
            throw 'Unable to log-on the server "' + server + '" with credentials ' + username + '/' + password + ' (incompatible mirth version?): ' + ex.message;
        }
        
        // check if login was successful
        if (loginStatus.getStatus() != com.mirth.connect.model.LoginStatus.Status.SUCCESS) {
            logger.error('Unable to log-on the server "' + server + '" with credentials ' + username + '/' + password + '(status ' + loginStatus.getStatus() + ')');
            return;
        }
    
            try {
            // get the server configuration
            var configuration = client.getServerConfiguration();
            // get the current date as string
            var backupDate = new String(DateUtil.getCurrentDate('yyyy-MM-dd HH:mm:ss'));
            var todaysFolder = new String(DateUtil.getCurrentDate('yyyy-MM-dd'));
    
            // check if user has appropriate permissions to write to the backup folder
            if(!java.nio.file.Files.isWritable(java.nio.file.Paths.get(backupFolder))){
                logger.error('No permissions to write to folder "' + backupFolder + '"');
                throw 'No permissions to write to folder "' + backupFolder + '"';
            }
            
            // generate the complete backupPath of the backup file
            var backupFolder = (new String(backupFolder)).replace(/\\/g, '/') + '/' + todaysFolder;
            var backupPath = backupFolder + '/' + server.replace(/:/g, '-') + '_' + backupDate.replace(/:/g, '-') + '.zip';
            // set the date of the backup in the server configuration
            configuration.setDate(backupDate);
            
            // create the directory if not existant
            org.apache.commons.io.FileUtils.forceMkdir(new java.io.File(backupFolder));
            // create the archive
            var archive = new net.lingala.zip4j.core.ZipFile(backupPath);
            // and set the compression parameters
            var archiveParameters = new net.lingala.zip4j.model.ZipParameters();
            archiveParameters.setCompressionMethod(net.lingala.zip4j.util.Zip4jConstants.COMP_DEFLATE);
            // and set the highest (and slowest) compression rate possible
            archiveParameters.setCompressionLevel(net.lingala.zip4j.util.Zip4jConstants.DEFLATE_LEVEL_ULTRA);
            // indicate that thei files will be streamed to the archive
            archiveParameters.setSourceExternalStream(true);
            
            // if a password was provided, encrypt the archive
            if ((archivePassword !== undefined) && archivePassword) {
                // activate encryption if password for archive is set
                archiveParameters.setEncryptFiles(true);
                // set the encryption algorithm
                archiveParameters.setEncryptionMethod(net.lingala.zip4j.util.Zip4jConstants.ENC_METHOD_AES);
                // and also the encrpytion strength
                archiveParameters.setAesKeyStrength(net.lingala.zip4j.util.Zip4jConstants.AES_STRENGTH_256);
                // set the password of the encrypted archive
                archiveParameters.setPassword(archivePassword);
            }
            
            // 1.) export the mirth configuration to the archive
            // create an xml representation of the configuration object
            configuration = serializer.serialize(configuration);
            // create a streem from the xml
            configuration = new java.io.ByteArrayInputStream(configuration.getBytes());
            logger.info('Exporting configuration of mirth server "' + server + '"');
            // set the filename for the server configuration in the archive
            archiveParameters.setFileNameInZip(server + '_' + backupDate.replace(/:/g, '-') + '.xml');
            // and write the file to the archive
            archive.addStream(configuration, archiveParameters);
    
            //2.) export the configuration map to the archive 
            // set the filename for the server configuration in the archive
            archiveParameters.setFileNameInZip('configuration.properties');
            var configMap = getConfigurationProperties(client);
            // and write the file to the archive
            archive.addStream(configMap, archiveParameters);
    
            // end the session
            client.logout();
            // and close the client instance
            client.close();
            logger.info('Configuration of mirth server "' + server + '" has been exported to "' + backupPath + '"');
        } catch (ex) {
            logger.error('unable to write file "' + backupPath + '": ' + ex.message);
        } finally{
            try{file.close();}catch(e){}
            try{configMap.close();}catch(e){}
        }
    }
    
    
    /**
        Provides the configuration map as an imput stream.
    */
    function getConfigurationProperties(client){
        // prepare structure
        var properties = new org.apache.commons.configuration.PropertiesConfiguration();
        // no fancy parsing here just a standard container
        properties.setDelimiterParsingDisabled(true);
        properties.setListDelimiter(0);
        properties.clear();
        var layout = properties.getLayout();
    
        // order the properties - basically just like the mirth admin does
        var sortedMap = java.util.TreeMap(java.lang.String.CASE_INSENSITIVE_ORDER);
        sortedMap.putAll(client.getConfigurationMap());
    
        // change the layout for obtainin
        for (var iterator = sortedMap.entrySet().iterator(); iterator.hasNext();) {
            var entry = iterator.next();
    
            var key = entry.getKey();
            // if the key is left emty, this entry is invalid and therefore skipped
            if(!key){continue;}
            
            var value = entry.getValue().getValue();
            var comment = entry.getValue().getComment();
            properties.setProperty(key, value);
            
            layout.setComment(key, comment ? comment : null);
        }
        
        // now write the file to a stream. Let's do everything on the fly
        var exportMap = new java.io.ByteArrayOutputStream();
        properties.save(exportMap);
    
        // provide an input stream that can directly be written to the archive
        return new java.io.ByteArrayInputStream(exportMap.toByteArray());
    }
    Attached Files
    Last edited by odo; 09-05-2019, 02:48 AM. Reason: Fixed copy/paste issue that led to an invalid java object path

  • #2
    Keep your backup folder clean

    Purpose:
    Intelligently remove outdated backups from your backup folder to avoid exceeding the capacity of the backup drive.

    This function should be called in the backup channel after all Mirth servers have been backuped.

    Parameters:
    • path
      The path to the directory that should be cleaned up. (This usually is the path of the backup folder)
    • maxFileAge
      Subdirectories that are older than the here specified number of days (1st day is today) will be deleted if number of resulting directories is not below the minimum specified by minNumberOfBackups
    • minNumberOfBackups
      The minimum number of backups that should be kept even if the maximum file age is exceeded (optional)


    Examples:
    Remove all backups that are older than 40 days:
    Code:
    cleanupDirectory('c:\temp', 40);
    Remove all backups that are older than 20 days but retain at least the 10 last backups:
    Code:
    cleanupDirectory('c:\temp', 20, 10);
    How to use:
    Copy the code below or import the attached code template:
    Code:
    /**
    	Removes all subdirectories from a given directory that are older than a given delay if a mimimal number of subdirectories is reached.<br/>
    	<br/>
    	<i>This function is used to for house keeping of backup folders.</i>
    
    	@param {String} path - the path to the directory that should be cleaned up
    	@param {Number} maxFileAge - Subdirectories that are older than the here specified number of days (1st day is today) will be deleted if number of resulting directories is not below the minimum specified by <i><b>minNumberOfBackups</b></i>
    	@param {Number} minNumberOfBackups - The minimum number of backups that should be kept even if the maximum file age is exceeded <i>(optional)</i>
    */
    function cleanupDirectory(path, maxFileAge, minFiles){
    
    	logger.info('Starting cleanup of directory "' + path + '".');
    
    	if(maxFileAge === undefined){
    		maxFileAge = 0;
    	} else if(maxFileAge > 0){
    		// as day 1 is today
    		maxFileAge--;
    	}
    	
    	// open path that should be scanned
    	var directory = new java.io.File(path);
    	// just directories are of interest here
    	content =  org.apache.commons.io.FileUtils.listFilesAndDirs(directory, org.apache.commons.io.filefilter.DirectoryFileFilter.DIRECTORY, org.apache.commons.io.filefilter.DirectoryFileFilter.DIRECTORY);	
    	// remove the parent dir from the result list
    	content.remove(directory);
    	// and transform the collection to an array
    	content = content.toArray();
    	logger.debug('Found ' + content.length  + ' subfolder in directory (' + content[1].getPath() + ')');
    	// sort list of fetched directories having the youngest on top (which should be )
    	java.util.Arrays.sort(content, org.apache.commons.io.comparator.LastModifiedFileComparator.LASTMODIFIED_REVERSE);
    	// calculate the threshold date
    	var dateThreshold =  new Date();
    	// use midnight as reference time
    	dateThreshold.setHours(0,0,0,0);
    	// and calculate the threshold date
    	dateThreshold.setDate(dateThreshold.getDate() - maxFileAge);
    	// in miliseconds
    	dateThreshold = dateThreshold.getTime();
    	logger.trace('Threshold: ' + dateThreshold); 
    	// assure number of subdirectories threshold is initialized
    	if(minFiles === undefined){
    		minFiles = 0;
    	}
    
    	// start from the end of the array
    	var index = content.length - 1;
    	// now remove the oldest directories if there are more than configured in 'minfiles'. 
    	// Take into account the the containg folder is also part of the array. Thus index has to be larger as minFiles
    	while((index >= minFiles) && (content[index].lastModified() < dateThreshold)){
    		// delete the sub directory
    		logger.info('Removing backup "' + content[index].getPath() + '"');
    		org.apache.commons.io.FileUtils.deleteDirectory(content[index]);
    		// and move on to the next directory
    		index--;
    	}
    	// if debug mode is active, also log the backups that remain
    	if(logger.isDebugEnabled()){
    		while(index >= 0){
    			logger.debug('Keeping backup "' + content[index].getPath() + '"');
    			index--;
    		}
    	}
    	
    	logger.info('Cleanup of directory "' + path + '" completed.');
    }
    Attached Files
    Last edited by odo; 05-18-2017, 01:25 AM.

    Comment


    • #3
      Really cool! Just to add to the conversation, Mirth Appliances also have automatic backup features that can include the MC installation directory, the server configuration (like you're doing), and even the entire MC database.
      Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

      Nicholas Rupley
      Work: 949-237-6069
      Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


      - How do I foo?
      - You just bar.

      Comment


      • #4
        Originally posted by narupley View Post
        Mirth Appliances also have automatic backup features that can include the MC installation directory, the server configuration (like you're doing), and even the entire MC database.
        What is "Mirth Appliance"?

        Comment


        • #5
          Originally posted by odo View Post
          What is "Mirth Appliance"?
          It's a hosting platform with Mirth Connect, pre-tuned PostgreSQL, and everything you need already installed and ready to go. It's much more than just a Linux image with MC on it though. It also has built-in load balancing, database replication, clustering, SNMP, SFTP server, SSL tunnels, CUPS printing, VPN services, SMTP relay, automatic backups, and most importantly an easy-to-use automatic upgrade service for our products (including Mirth Connect). All of that and more through a web-based consolidated control panel. It can be deployed on your own hardware with a virtual image, or hosted by us.

          There's a user / deployment guide here: https://www.mirthcorp.com/community/...ageId=18579907
          And a brochure on deployment options here: http://bridge.mirth.com/media/3120/n...tions-fl27.pdf
          Last edited by narupley; 05-18-2017, 12:42 PM.
          Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

          Nicholas Rupley
          Work: 949-237-6069
          Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


          - How do I foo?
          - You just bar.

          Comment


          • #6
            Thanks for this! I had done it previously using a batch script, but this will work nicely.

            Comment


            • #7
              Hello,

              Thanks for the nice work. It works well at the version 3.5.1.

              Best Regards,

              Haluk Celikel

              Comment


              • #8
                Originally posted by haluk View Post
                It works well at the version 3.5.1.l
                Great to hear - thanx for the feedback!

                Comment


                • #9
                  Thanks for sharing your backup process - this is fantastic!

                  I see that you are backing up the Configuration but do you also backup up your Configuration Map (Export Map)?

                  Thanks again,
                  David
                  Last edited by davidap; 01-18-2018, 01:38 PM.

                  Comment


                  • #10
                    Originally posted by davidap View Post
                    I see that you are backing up the Configuration but do you also backup up your Configuration Map (Export Map)?
                    No, so far all configuration that is stored in the db is backed up. The configuration map is an external file. However, it is read to memory and thus it should be possible to also back it up remotely.

                    I'll have a look into that as soon as I get some time and update this thread with my findings.

                    Comment


                    • #11
                      I was able to add the export of the configuration map to your function by adding:

                      var configMap = client.getConfigurationMap();

                      and related lines to write it out the same way you did with the Mirth configuration.

                      I just have to figure out how to encrypt the map file now since our map has a password I need to protect.

                      Comment


                      • #12
                        Did it work? Writing the configuration map is different from writing the Mirth configuration as it is no xml but a property map (with comments).

                        As promised, I extended the backup functionality:
                        • Added configuration map export
                        • Added zip compression of the backup (resulting size is now 5-10% of the size before)
                        • Added optional 256 bit AES encryption of the export


                        The updated code template and all info for using it can be found in the 1st post.
                        Last edited by odo; 01-25-2018, 12:28 AM.

                        Comment


                        • #13
                          &quot;archivePassword&quot; is not defined

                          [2018-01-24 17:10:56,674] ERROR (transformer:?): unable to write file "C:/Users/....../Mirth/Backup Channels//2018-01-24/localhost_2018-01-24 17-10-56.zip": "archivePassword" is not defined.

                          got this error when not specifying the archivePassword.
                          Even if the function call doesnt require it.
                          function backupMirthServer(username, password, server, backupFolder)

                          If I add it to the function and call:
                          function backupMirthServer(username, password, server, backupFolder, archivePassword)

                          This error is not shown anymore.
                          Seems
                          Code:
                          if (archivePassword !== undefined) {
                          doesnt do the trick.
                          X Connections
                          https://documentor.email
                          https://www.x-connections.com

                          Comment


                          • #14
                            &quot;ByteArrayOutputStream&quot; is not defined.

                            Btw Im using 3.5.1 on windows

                            Any idea what causes this?

                            Exporting configuration of mirth server "127.0.0.1"
                            [2018-01-24 17:22:42,369] ERROR (transformer:?): unable to write file "C:/Users/...................Mirth/Backup Channels//2018-01-24/127.0.0.1_2018-01-24 17-22-39.zip": "ByteArrayOutputStream" is not defined.
                            X Connections
                            https://documentor.email
                            https://www.x-connections.com

                            Comment


                            • #15
                              solved

                              I added in: function getConfigurationProperties(client) java.io

                              Code:
                              // now write the file to a stream. Let's do everything on the fly
                              	var exportMap = new java.io.ByteArrayOutputStream();
                              X Connections
                              https://documentor.email
                              https://www.x-connections.com

                              Comment

                              Working...
                              X