Announcement

Collapse
No announcement yet.

Unzip transformation on file data

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Unzip transformation on file data

    Hi,

    We are getting a bunch of zipped files over FTP.. we have to unzip them and move the unzipped files to a target location.

    I pick the files on a src connector.. I have a Java method that does unzipping, and returns the unzipped bytes. I have to set these bytes on the current message, so that it gets moved to target as unzipped content.

    I tried a couple ways in my Javascript writer as below, both didnt work:

    1. msg.setTransformedData(byteData)
    --This one errors as - "ReferenceError: "msg" is not defined."
    I was guessing msg is more of a global variable in the setup.

    2. channelMap.put('unzippedData', byteData)
    -- This one doesnt throw any errors per se, but doesnt write the unzipped data onto the file through the filewriter. I tried mapping the channelMap.get('unzippedData') to a transformer variable 'transContent', and used ${transContent} in the FileWriter, but thats corrupting data. I get a file with 10 bytes, for a 35k data.

    Any other suggestions? or any way to get one of the above to work?
    Let me know.

    I have tried the below, and the file gets written fine, but its a local file, and I want to write to a remote one:
    FileUtil.write('unziptest.txt', false, byteData);

    I'm on Mirth Connect 1.8.2.


    Thank You!
    Deepa

  • #2
    Originally posted by deepa View Post
    Hi,
    1. msg.setTransformedData(byteData)
    --This one errors as - "ReferenceError: "msg" is not defined."
    I was guessing msg is more of a global variable in the setup.

    I tried the message.setTransformedData(byteData) here, and that says setTransformedData() is not a valid operation on message data.

    I am surely hoping there is an easy way to get this working. Please pour in your suggestions. Thanks!

    Comment


    • #3
      I have planned this kind of operation fo the near future, but I did not yet start developing it.
      My idea how do do is another one:
      Pick the file from the FTP server an put it in some directory
      Then, on a postprocessor step, unzip the file with
      Code:
      java.lang.Runtime.getRuntime().exec("system_command");
      or with a java function, if I can find one.
      JJ Mouris
      FFE Luxembourg
      IT-Department

      Comment


      • #4
        Thanks for writing back Mouris.

        I am unzipping data in a Java program with ZipInputStream, and am calling that method from the Mirth's Javascript Writer. So, the unzipping part is in place. I can also write it to a local file successfully using FileUtil.write().

        But I want to set the unzipped bytes on the channel's message, or on a channel variable, so that it gets written to the remote location.

        Comment


        • #5
          Hi deepa,

          For option #2, did you try setting the file type to Binary (the default is ASCII)?
          Gerald Bortis | Chief Information Officer | Mirth Corporation

          Comment


          • #6
            Yes, I did try with the filetype set to both ASCII and binary..

            ..but actually, the source type is binary, and the target type is ASCII. The unzipped bytes are plain text.

            Comment


            • #7
              Since the File Writer needs a String, you will probably need to convert the byte[] returned from the unzipping method to a String. Something like:

              Code:
              channelMap.put("unzippedData", new String(byteData));
              Gerald Bortis | Chief Information Officer | Mirth Corporation

              Comment


              • #8
                Nope, didnt seem to work.

                I am attaching my test channel, for your reference. I guess that should make it easier.
                Attached Files

                Comment


                • #9
                  1. Create a channel with Source File Reader, with File Type set to Binary. This way the ZIP file is loaded and converted into BASE64 string and sent to Destination.
                  2. Select JavaScript Writer for Destination, and the following source code:


                  ////////////////////////////
                  // SETTINGS
                  ////////////////////////////
                  // ZIP file input directory
                  var PATHDESTDIR = "C:\\tmp\\mirth\\out";
                  var BUFFER = 1024;


                  ////////////////////////////
                  // MAIN BODY
                  ////////////////////////////

                  // decode rawData from BASE64 to a byte array
                  var strBase64Data = messageObject.getRawData();
                  var decodedBytes = FileUtil.decode(strBase64Data);

                  // process all zipped files
                  var is = new java.io.ByteArrayInputStream(decodedBytes);
                  var zis = new java.util.zip.ZipInputStream(is);

                  var entry;
                  while((entry = zis.getNextEntry()) != null) {

                  // save file
                  var count;
                  var buffer = java.lang.reflect.Array.newInstance(java.lang.Byte .TYPE, BUFFER);

                  var fileOut = new java.io.File(PATHDESTDIR + "\\" + entry.getName());
                  var fos = new java.io.FileOutputStream(fileOut);

                  // read byte content from zipped file
                  while ((count = zis.read(buffer, 0, BUFFER)) != -1) {
                  fos.write(buffer, 0, count);
                  }

                  fos.close();
                  }
                  zis.close();
                  Last edited by reljicb; 05-24-2012, 12:30 AM.

                  Comment


                  • #10
                    Another example:

                    http://www.mirthcorp.com/community/f...ead.php?t=7016

                    Originally posted by narupley View Post
                    If you want to send a ZIP file, it's as easy as sending its Base64 representation using a File Writer in binary mode.

                    It sounds like you're trying to extract a ZIP file though right? If so, the java.util.zip package is your friend. For example, I just set up a File Reader channel to read in a ZIP file (in binary mode of course), and pass an XML structure containing all of the extracted files to the destination connector. Here's the code I used for the source transformer:

                    Code:
                    importPackage(java.io);
                    importPackage(java.util.zip);
                    
                    var zis = new ZipInputStream(new ByteArrayInputStream(org.apache.commons.codec.binary.Base64.decodeBase64(messageObject.getRawData().replaceAll('[^0-9a-zA-Z\\+\\/\\=]',''))));
                    var bufSize = 1024;
                    
                    var file;
                    while (file = zis.getNextEntry()) {
                    	var bos = new ByteArrayOutputStream();
                    	var data = getBlankByteArray(bufSize);
                    	var len = -1;
                    	while ((len = zis.read(data,0,bufSize)) != -1)
                    		bos.write(data,0,len);
                    	var fileNode = <file/>;
                    	fileNode.name = file.getName();
                    	fileNode.content =  new java.lang.String(bos.toByteArray());
                    	tmp.appendChild(fileNode);
                    } 
                    zis.close();
                    
                    function getBlankByteArray(length) {
                    	var bos = new java.io.ByteArrayOutputStream();
                    	for (var i = 1; i <= length; i++)
                    		bos.write(0);
                    	return bos.toByteArray();
                    }
                    I used Delimited Text for my source inbound protocol, XML for the outbound, and "<files/>" for the outbound template. That code doesn't take charsets into consideration, so feel free to add that to the String constructor if necessary.
                    Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

                    Nicholas Rupley
                    Work: 949-237-6069
                    Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


                    - How do I foo?
                    - You just bar.

                    Comment

                    Working...
                    X