Over the last 3 weeks I’ve virtually become an EDI expert. Believe me I have no desire to be an EDI expert. It’s an ugly protocol.
For those who do not know, billions of dollars of business are done through EDI. Health care claims, supply chain management between a retailer (like Walmart) and a supplier (like Frito-Lay), these are just two examples of what EDI can do. EDI is essentially like XML in that it defines a way to lay out units of data.
XML: <foo name=”blah”>bar</foo>
EDI: FOO*BLAH*BAR~
The problem is that EDI is very linear and has a hacked up scheme to add element nesting. Representing a foo object with several child bar objects is trivial in XML:
<foo>
<bar/>
<bar/>
<bar/>
</foo>
But problematic in EDI:
FOO*123*456~
BAR*1~
BAR*2~
BAR*3~
BLAH*7~
with the obvious problem being, how does the parser know what level BLAH belongs to? Is it a child of FOO like the BAR lines or is it a sibling? There’s no way a generic parser can tell - you need a parser specific to each transaction.
So for the past three weeks I have done three things:
1) Create a JavaCC model of a SEF file. The SEF file describes the format of an EDI transaction. Here’s an example of a SEF file which describes a set of health care claims.
2) The JavaCC model creates a parser for the SEF file which feeds data to a generic EDI object model. This object model knows about generic EDI data structures and SEF file describes those data structures and how to tell the file structure.
3) The generic EDI object model creates a transaction parser and object model for that EDI transaction which specifically knows how to read/write itself to/from an EDI file. For the 837 transaction, it creates about 400 java files with 700k of code! Currently the generated 837 parser can parse a file with 1600 claims (about 60k lines or 1.5MB) in 3.5 seconds. Not too bad for completely unoptimized code.
The standards bodies which create these EDI standards are obviously moving toward XML as the way of the future however they are extremely slow-moving so it could be years before anything is finalized. For the next 5-10 years, EDI is here to stay.
So you may ask, why am I working on this? Well, Webify has a set of applications which deal with health care claims but until now we have treated the claims file as a black box; the doctor uploaded it to us and we sent it to the insurance company. But to really add value we need to be able to allow online editing of the claims, add/remove erroneous claims, validate claims, etc. To do this, we need to be able to “open” that black box and understand its contents. These EDI transaction object models are the underpinning for that intelligence. Once we can parse the claims and other health care transactions into java objects, the sky’s the limit as to what we can do with it.