problem with Umlaut characters

[svart]

hi,

I've got a problem with Umlauts in a preprocessing script.
the channel reads a csv file. a preprocessing script is supposed to convert it to xml.
when the csv contains umlauts like 'ü', they appear in the 'View message->Raw message' tab as '?'.
I've even tries to replace umlauts with other characters like 'ue'. this doesn't work, so I think the problem might appear before the message gets into the preprocessor.

the different encodings in the channel source didn't help either.

message source (csv) :

123;14.05.2007;0;123456;Mustermann;Maria;Überweg 5;54321;Köln

raw message in mirth:

    <Patient>
        <rgsnumber>123</rgsnumber>
        <date>14.05.2007</date>
        <bet>0</bet>
        <case_number>123456</case_number>
        <name>Mustermann</name>
        <sname>Maria</sname>
        <street>?berweg 5</street>
        <ar>54321</ar>
        <city>K?ln</city>
    </Patient>

How can I make sure to keep the umlauts ?

[albertosaez]

Reroblem with Umlaut characters

First of all ¿Wich encoding are you using in your CVS file? To discover it, you can use the "file" command if you're under linux (or windows using cygwin) or with an editor like Jedit.

The problem you're talking about is related with the viewer, not with the Mirth engine. Mirth, in its internal process can handle different encodings, but the Mirth console doesn't show it well.

To test if Mirth is woking fine with this characters try the next in your channel: output the XML you've transformed to a file writter, writting the XML. Then try open your xml with IE, Firefox, or a XML editor (Oxygen, XMLSpy, etc.)

If this doen't works, plese, post an example of your CSV file, and your channel.

[svart]

Reroblem with Umlaut characters

file encoding is ISO-8859 text
I've set up a file writer that writes out the transformed xml as you suggested. the characters in that file are not showing correctly, they appear as '?'. I viewed them in ultra-edit which is usually able to show any encoding.


Here's my csv:

623;14.05.2007;0;12345;Wohl;Andreas;Freepsum 4;48596;Krummhörn
624;14.05.2007;0;45678;Bök;Gudrun;West 14A;40596;Barssel
625;14.05.2007;0;43214;Bölsch;Armin;Ahlener 12;09876;Dörpen
626;14.05.2007;0;54678;Fedder;Frank;Wurmweg 4;50968;Köln

Here's my channel:

<com.webreach.mirth.model.Channel>
  <id>335aad43-014a-44b5-81b6-181e724a40ed</id>
  <name>Hi_in_router_out</name>
  <description></description>
  <enabled>true</enabled>
  <version>1.4.0</version>
  <revision>41</revision>
  <sourceConnector>
    <name>sourceConnector</name>
    <properties>
      <property name="fileAge">0</property>
      <property name="autoDelete">1</property>
      <property name="moveToDirectory">C:/backup</property>
      <property name="pollingFrequency">600000</property>
      <property name="host">c:/incoming</property>
      <property name="moveToPattern">${DATE}-${SYSTIME}-${ORIGINALNAME}</property>
      <property name="checkFileAge">0</property>
      <property name="DataType">File Reader</property>
      <property name="processBatchFiles">1</property>
      <property name="charsetEncoding">DEFAULT_ENCODING</property>
      <property name="sortAttribute">date</property>
      <property name="binary">0</property>
      <property name="fileFilter">RG.csv</property>
    </properties>
    <transformer>
      <steps>
        <com.webreach.mirth.model.Step>
          <sequenceNumber>0</sequenceNumber>
          <name>Router</name>
          <script>for each (message in msg[&apos;Patient&apos;]){
router.routeMessage(&quot;Next_Channel&quot;, message);
}</script>
          <type>JavaScript</type>
          <data class="map">
            <entry>
              <string>Script</string>
              <string>for each (message in msg[&apos;Patient&apos;]){
router.routeMessage(&quot;Next_Channel&quot;, message);
}</string>
            </entry>
          </data>
        </com.webreach.mirth.model.Step>
      </steps>
      <inboundTemplate>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&#x0D;&lt;!--  Generated using Mirth PreProc   --&gt;&#x0D;&lt;Hi_in&gt;&#x0D;    &lt;Patient&gt;&#x0D;        &lt;rechnungsnummer&gt;rechungsnummer&lt;/rechnungsnummer&gt;&#x0D;        &lt;datum&gt;datum&lt;/datum&gt;&#x0D;        &lt;betrag&gt;betrag&lt;/betrag&gt;&#x0D;        &lt;fall_nummer&gt;p_nummer&lt;/fall_nummer&gt;&#x0D;        &lt;name&gt;name&lt;/name&gt;&#x0D;        &lt;vorname&gt;vorname&lt;/vorname&gt;&#x0D;        &lt;strasse&gt;strasse&lt;/strasse&gt;&#x0D;        &lt;plz&gt;plz&lt;/plz&gt;&#x0D;        &lt;ort&gt;ort&lt;/ort&gt;&#x0D;    &lt;/Patient&gt;&#x0D;&lt;/Hi_in&gt;</inboundTemplate>
      <outboundTemplate></outboundTemplate>
      <inboundProtocol>XML</inboundProtocol>
      <outboundProtocol>XML</outboundProtocol>
      <inboundProperties/>
      <outboundProperties/>
    </transformer>
    <filter>
      <rules/>
    </filter>
    <transportName>File Reader</transportName>
    <mode>SOURCE</mode>
  </sourceConnector>
  <destinationConnectors>
    <com.webreach.mirth.model.Connector>
      <name>Destination 1</name>
      <properties>
        <property name="host">sink</property>
        <property name="channelName">None</property>
        <property name="DataType">Channel Writer</property>
      </properties>
      <transformer>
        <steps/>
        <inboundTemplate></inboundTemplate>
        <outboundTemplate></outboundTemplate>
        <inboundProtocol>XML</inboundProtocol>
        <outboundProtocol>XML</outboundProtocol>
        <inboundProperties/>
        <outboundProperties/>
      </transformer>
      <filter>
        <rules/>
      </filter>
      <transportName>Channel Writer</transportName>
      <mode>DESTINATION</mode>
    </com.webreach.mirth.model.Connector>
  </destinationConnectors>
  <properties>
    <property name="synchronous">true</property>
    <property name="encryptData">false</property>
    <property name="dont_store_filtered">false</property>
    <property name="store_messages">true</property>
    <property name="initialState">started</property>
    <property name="max_message_age">-1</property>
    <property name="transactional">false</property>
    <property name="error_messages_only">false</property>
  </properties>
  <preprocessingScript>message = message.replace(/[\\r\\n]/g, &quot;\\n&quot;«»);
message = replaceUmlauts(message, 0);

var columnnames = new Array(&quot;rechnungsnummer&quot;,&quot;datum&quot;,&quot;betrag&quot;,&quot;fall_nummer&quot;,&quot;name&quot;,&quot;vorname&quot;,&quot;strasse&quot;,&quot;plz&quot;,&quot;ort&quot;«»);
var nofcolumns = 9;
var lines = new Array();
var newmessage = new String();
var i;
newmessage = &quot;&lt;!--  Generated using Mirth PreProc   --&gt;\\n&quot;; 
newmessage += &quot;&lt;Hi_in&gt;\\n&quot;;
rawData = message;

// Auskommentierte Zeilen für mehrere Zeilen pro File
 lines = rawData.split(&quot;\\n&quot;«»); 
for (i=0;i&lt;lines.length;i++) {
	currline = lines[i];

	newmessage += &quot;&lt;Patient&gt;\\n&quot;;
	var data = new Array();
	data = currline.split(&quot;;&quot;«»);

//	data = rawData.split(&quot;;&quot;«»);
	for (var j = 0; j &lt; nofcolumns; j++){
		newmessage += &quot;&lt;&quot; + columnnames[j] + &quot;&gt;&quot; + data[j] + &quot;&lt;/&quot; + columnnames[j] + &quot;&gt;\\n&quot;;
}
	newmessage += &quot;&lt;/Patient&gt;\\n&quot;;
 }

newmessage += &quot;&lt;/Hi_in&gt;\\n&quot;;
return newmessage

function replaceUmlauts(string, index){
		    var anArray = new Array(2);
		    anArray[0] = new Array(&quot;Ö&quot;, &quot;ö&quot;, &quot;Ä&quot;, &quot;ä&quot;, &quot;Ü&quot;, &quot;ü&quot;, &quot;ß&quot;«»);
		    anArray[1] = new Array(&quot;Oe&quot;, &quot;oe&quot;, &quot;Ae&quot;, &quot;ae&quot;, &quot;Ue&quot;, &quot;ue&quot;, &quot;sz&quot;«»);
		    
		    for (var i=0; i&lt;anArray[index].length; i++){
				myRegExp = new RegExp(anArray[index][i],&quot;g&quot;«»);
				string = string.replace(myRegExp, anArray[(index==0?1:0)][i]);
			}
		    return string;
		}
</preprocessingScript>
</com.webreach.mirth.model.Channel>

[Java4Fun]

i've got the same problem. And havn't found a solution till now.
is there any solution to solve this?

Thanks.
Bernhard

[jacobb]

This may have been resolved in http://www.mirthcorp.com/community/i...wse/MIRTH-1872

If you were using the Send Message feature to manually process a message, Mirth Connect 2.1.0 would not preserve the special characters.