No announcement yet.

Multiple source folder SFTP connection

  • Filter
  • Time
  • Show
Clear All
new posts

  • Multiple source folder SFTP connection

    Wondering if there is a good way to connect to multiple sftp SOURCE folders? Please note again, this is for a source connection, and not the destination.

    For example, the SFTP folder were structured as follows:

    SFTP/Main/FolderA and SFTP/Main/FolderB

    The mirth channel should pull files every 24 hours. And if there are files in SFTP/Main/FolderA, pull files from that folder. If not, then pull files from SFTP/Main/FolderB.

    Any help on a good way to do this would be appreciated.

  • #2
    If FolderA and FolderB are the only subfolders of Main, simply use SFTP/Main as file location and check the "Include All Subdirectories"-Option of the sftp-connector.


    • #3
      @ODO - The issue with that is that it will end up reading from both folders. We need to make sure that it tries to read from the first folder before it reads from the second folder. If there are no files in the first folder, then, and ONLY THEN, should it read from the second folder.


      • #4
        Use the fileDirectory attribute in the source map to determine where the file is coming from. The pollComplete attribute will tell you if the current message is the last one that has been polled.

        I would try the following:

        1. create a source filter that writes the current message in a globalChannelMap variable e.g. messageToProcess.
          • If fileDirectory does not yet exist and fileDirectory=folder B ==> messageToProcess=currentMessage, otherwise do nothing.
          • If fileDirectory=folder A ==> messageToProcess=yourMessage.
        2. If current message has the pollComplete-flag set ==> send messageToProcess to destination, otherwise skip.


        • #5
          If you're comfortable working in javascript (I think there may be some examples in this forum of using either the mirth SftpConnection class or the Jsch library directly) you could use a javascript reader instead of a file reader.

          This would allow you to check if directory A contains file before looking to directory B.

          If the number of files are few or the filesizes are small, you could read all of the files from your target directory in the javascript reader and create messages similar to how the file reader would. The major differences would be (1) you need to be able to fit all of the files in memory at once, and (2) you need to manage deleting or moving the source files yourself after reading them. You might want to consider waiting to delete or move until the post-processor. There could be cases if you are deleting immediately after reading that you have an error that causes a loss of data before it's committed to the database.

          Otherwise, if the total file size is expected to be large, you could do a directory listing in the javascript reader, and pass the path to each file as the message. Then in the transformer you could download and replace the path in the message with the actual contents.