Tweet
HI All,
Trying to figure out how to do a rather simple task, but running into a wall. The task itself is this:
- we need to read a csv file with some data lines in it, one of the fields is a pdf filename that then needs to be read either as Base64 (or binary and then converted to Base64) from the same sftp folder to be associated with the csv record read earlier.
For the life of me I can't figure out how to do it all in one ftp reader channel.
Current solution is to set Filename Filter Pattern to "*.pdf", read all pdfs as base 64 (binary), then use a custom java sftp function to read (as text) and parse the csv file once sourceMap.get("pollComplete") == true. This works if there is at least one pdf, otherwise the channel quits before it gets to the destination which reads the csv.
Alternatively I can set Filename Filter Pattern to *.csv to read the csv as text, but then there appears to be no way to send extracted filename to sftp reader to get that particular file. Tried to use custom java sfto function to read pdfs in a destination, that works, but then converting read bites to Base64 (used FileUtil.decode(data); for that) doesn't appear to work.
Is there a way to read csv and then send read lines to an sftp channel line by line for it to extract pdf filename from input and use that as Filename Filter Pattern to read one particular file? I think just sending the filename would work for us as well.
Thank you, any input would be greatly appreciated!
P.S. I can set Filename Filter Pattern to *.* and read pdfs and csv in one go, but then it isn't clear how to convert csv from binary to text so I can parse the lines in it...
Trying to figure out how to do a rather simple task, but running into a wall. The task itself is this:
- we need to read a csv file with some data lines in it, one of the fields is a pdf filename that then needs to be read either as Base64 (or binary and then converted to Base64) from the same sftp folder to be associated with the csv record read earlier.
For the life of me I can't figure out how to do it all in one ftp reader channel.
Current solution is to set Filename Filter Pattern to "*.pdf", read all pdfs as base 64 (binary), then use a custom java sftp function to read (as text) and parse the csv file once sourceMap.get("pollComplete") == true. This works if there is at least one pdf, otherwise the channel quits before it gets to the destination which reads the csv.
Alternatively I can set Filename Filter Pattern to *.csv to read the csv as text, but then there appears to be no way to send extracted filename to sftp reader to get that particular file. Tried to use custom java sfto function to read pdfs in a destination, that works, but then converting read bites to Base64 (used FileUtil.decode(data); for that) doesn't appear to work.
Is there a way to read csv and then send read lines to an sftp channel line by line for it to extract pdf filename from input and use that as Filename Filter Pattern to read one particular file? I think just sending the filename would work for us as well.
Thank you, any input would be greatly appreciated!
P.S. I can set Filename Filter Pattern to *.* and read pdfs and csv in one go, but then it isn't clear how to convert csv from binary to text so I can parse the lines in it...
Comment