No announcement yet.

Is MC good solution for mapping different data models?

  • Filter
  • Time
  • Show
Clear All
new posts

  • Is MC good solution for mapping different data models?


    currently we are looking for a solution to easily map EMR data to several data models used in medical research, like:

    PCORnet Common Data Model (CDM) and HMORN Virtual Data Warehouse. About 50% of fields in these models are very similar, so it would make sense to take PCORnet data model and just extend it to have other data models ready.

    At this time we use MS SQL queries and procedures to do this job, but in many cases it is very time consuming and requires extensive programing knowledge. After data is prepared we then have to manually write code to convert data into HL7 format. So the process is:

    EMR --> HL7 --> Data Warehouse --> mapping/conversion --> HL7 --> data submission

    When I was presented with Mirth, I saw many ways how it could save us time and make this mapping easier for the data managers/admins. Also we would not need to do any additional conversion back to HL7.

    I'm pretty new to this field, so I would like to hear back from experienced ones if that is the right tool to solve our problem.

    In my current understanding we could prepare Mirth Connect configs for mapping for each research data model, so we could use it in ways:

    EMR --> HL7 --> MC Data Model 1 --> data submission

    Or even reuse part of one data model's config to create other data models, that are loosely based on the first one.

    EMR --> HL7 --> MC Data Model 1 --> MC Data Model 2 --> data submission

    Thank you!
    Last edited by mtomas; 04-23-2015, 05:00 PM.

  • #2
    I've definitely used MC in the past to convert to a lot of weird data models before...

    First, a lot of formats I've come across are handled by the Delimited Text data type, because you can customize the column/record delimiters. So it will handle CSV, or tab-delimited, or pipe-delimited, etc. A lot of places I've worked with used some variation on a pipe-delimited text file, so Delimited Text was easy for that.

    There are cases where Delimited Text doesn't quite fit the bill, but in those cases you can just use the Raw data type, and use a transformer to parse it yourself. It does take some programming sometimes, but it's just JavaScript, which is one of the most widely-used languages out there (more and more actually due to frameworks like Node.js, AngularJS, etc.), and one of the most easily accessible / easily learned languages out there.

    For example, I remember working with a pipe-delimited format that actually had commas within each field, like sub-fields. Similar to how HL7 has fields (delimited by |) and components (delimited by ^), except quoting values with double quotes was also allowed. It was just weird enough that Delimited Text didn't quite do what I wanted, but I was able to fairly easily set up a transformer that iterated through each field and parse things into XML, etc.

    There were also a couple of other cases where a client already had a standard Java model for their objects. Server-side Java is still going really strong, stronger than C++ or C# even in 2015 ( So if there are model JARs they can give you, you can include them in MC, then invoke those classes/methods in JavaScript almost exactly like you would in Java. That's useful if the client has already done the parsing work, and all you have to do is invoke some utility class in your transformer to convert some raw data format to a model object, which you can then work with more easily.
    Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

    Nicholas Rupley
    Work: 949-237-6069
    Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.

    - How do I foo?
    - You just bar.


    • #3
      Thank you for the answer!

      It seems you talk about Data Model as if it is a platform to store data (csv, java, pipe-delimited, etc). And that is a possible situation.

      For our purposes 2 different data models can be stored in the same platform (i.e. MS SQL). Maybe you call data mapping to what we need?