240 likes | 328 Views
EDI to XML Using XML to take the headaches out of EDI processing. Blair Schneider McKay Clareos, Inc. http://www.clareos.com BlairSchneiderMcKay@comcast.net. Agenda. Project Motivation EDI Basics Syntactic Conversion: EDI to XML Semantic Conversion: XML to XML
E N D
EDI to XMLUsing XML to take the headaches out of EDI processing Blair Schneider McKay Clareos, Inc. http://www.clareos.com BlairSchneiderMcKay@comcast.net
Agenda • Project Motivation • EDI Basics • Syntactic Conversion: EDI to XML • Semantic Conversion: XML to XML • Validating the Input Data • Storing and Using the Results • System Instrumentation • Q & A Edi To Xml – Slide 2
Project Motivation AT&T Custom User Solutions (ACUS) • niche telecomm biller for colleges, military bases, state governments, other AT&T units, etc. • small development group. XP and agile development. • at issue was one of our VAS offerings: reconcile client facility bills and help them address discrepancies. The Issue: Reconciliation was done by manual comparison of electronic usage records to paper invoices. This was costly and slow, and it created an extra quality-control burden. Edi To Xml – Slide 3
Agenda • Project Motivation • EDI Basics • Syntactic Conversion: EDI to XML • Semantic Conversion: XML to XML • Validating the Input Data • Storing and Using the Results • System Instrumentation • Q & A Edi To Xml – Slide 4
Electronic Data Interchange • early alternative to facsimile and postal mail • standards-based - ANSI / ASC X12 / DISA - UN/EDIFACT - http://www.disa.org • hardware, data, software, and network specs “VANs”, mailboxes, wire-transfer protocols, etc. • widespread DISA estimates over 300,000 companies use it. • becoming more integrated with XML Example: “Electronic Business XML” (ebXML) Edi To Xml – Slide 5
Sample EDI Document ISA~00~ ~00~ ~01~00001123 ~ZZ~MICHIGANSTA ~040103~1950~U~00401~000001029~0~T~> GS~CI~006968523CRI~123456XYZ~20040407~1527~128~X~004010 ST~811~000000001 BIG~20040407~734555121299920040407~~~~~DI REF~EH~773 REF~12~7345551212999 REF~AP~7345551212999 ITD~18~3~~~~20040421 ...snip... SE~17921~10290002 GE~2~128 IEA~1~000001029 Modeled on samples in: http://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdf Edi To Xml – Slide 6
EDI Technical Summary ISA GS ST Details SE GE IEA data element types numeric, decimal, identifier, string, date, time delimiters data-element-separator first character after “ISA”, often “~” sub-element-separator first character of 16th ISA data element segment-terminator character after the sub-element-separator hierarchies segments nest in “loops” Edi To Xml – Slide 7
Agenda • Project Motivation • EDI Basics • Syntactic Conversion: EDI to XML • Semantic Conversion: XML to XML • Validating the Input Data • Storing and Using the Results • System Instrumentation • Q & A Edi To Xml – Slide 8
Why XML? • The 811 document is already hierarchical. • It’s easier. Many companies find EDI too expensive, too complex, and too inflexible. • Once something is in XML format, you can “go anywhere” with it. By converting EDI to XML you trade complex, “single-use” tools for generic XML tools like DTD, W3C Schema, XSLT, SOAP, DOM, SAX, etc. Edi To Xml – Slide 9
More Motivation… USD Segment If USD08 is populated, then the amount in USD08 is the segment amount. If USD08 is not populated, then the following determination is made to determine the segment amount: If USD05 = “M4”, then the amount in USD06 is the segment amount. If USD05 ¹ “M4”, then the product of USD03 and USD06 is the segment amount. If USD01 = “O”, then the segment amount is for information purposes only. The USD segment can be used to provide taper table information for WATS billing. In this use, USD01 would equal “O”. Note 1: When both USD03 and USD08 are populated, the amount in USD08 takes precedence and becomes the segment amount. Note 2: In order to verify amount in USD08, it is necessary to multiply USD03 * USD06 * USD07 * Percent of Interstate Usage. The Percent of Interstate Usage can be found in an SI pair (DE 1000 = ‘PU’) within the SLN loop, when it is present. From: http://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdf, p6.2.8 Edi To Xml – Slide 10
Sample EDI Document - Revisited ISA~00~ ~00~ ~01~00001123 ~ZZ~MICHIGANSTA ~040103~1950~U~00401~000001029~0~T~> GS~CI~006968523CRI~123456XYZ~20040407~1527~128~X~004010 ST~811~000000001 BIG~20040407~734555121299920040407~~~~~DI REF~EH~773 REF~12~7345551212999 REF~AP~7345551212999 ITD~18~3~~~~20040421 ...snip... SE~17921~10290002 GE~2~128 IEA~1~000001029 Modeled on samples in: http://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdf Edi To Xml – Slide 11
Raw XML <HL> <HL01>4</HL01> <HL02>3</HL02> <HL03>8</HL03> <HL04>1</HL04> <SLN> <SLN01>5.0</SLN01> <SLN02></SLN02> <SLN03>I</SLN03> <SLN04>M4</SLN04> <SLN05>11.85</SLN05> <SLN06></SLN06> <SLN07>I</SLN07> <SLN08>SV</SLN08> <SLN09>0705</SLN09> Edi2Xml.exe HL~4~3~8~1 SLN~5.0~~I~1~M4~11.85~~I~SV~0705 Edi To Xml – Slide 12
The Edi2Xml Utility class Segment { public: enum Relationship { child, sibling, uncle }; virtual Relationship comp( const Segment& other) const; // ... }; Sample init file: Nesting ISA 1 Nesting IEA 1 Nesting GS 2 Nesting GE 2 Nesting ST 3 Nesting SE 3 Nesting_FieldComp HL 4 3 Single * Edi To Xml – Slide 13
Agenda • Project Motivation • EDI Basics • Syntactic Conversion: EDI to XML • Semantic Conversion: XML to XML • Validating the Input Data • Storing and Using the Results • System Instrumentation • Q & A Edi To Xml – Slide 14
XSLT Examples <xsl:template match="ST[ST01='811']"> <Invoice> <xsl:apply-templates/> </Invoice> </xsl:template> <xsl:template name="conv_access_type"> <xsl:param name="cd"/> <xsl:choose> <xsl:when test="$cd='81'">Local</xsl:when> <xsl:when test="$cd='82'">IntraState/InterLata</xsl:when> <xsl:when test="$cd='83'">IntraState/IntraLata</xsl:when> ...snip... Edi To Xml – Slide 15
Translated XML – v.1 <HL> <HL01>4</HL01> <HL02>3</HL02> <HL03>8</HL03> <HL04>1</HL04> <SLN> <SLN01>5.0</SLN01> <SLN02></SLN02> <SLN03>I</SLN03> <SLN04>M4</SLN04> <SLN05>11.85</SLN05> <SLN06></SLN06> <SLN07>I</SLN07> <SLN08>SV</SLN08> <SLN09>0705</SLN09> <HL idx='4' parent='3' level='8' haschildren='yes'> <SLN id1='5.0' relcode='I' qty='1' type='money' included='yes' service-rendered='yes' svc_type='0705'> 11.85 </SLN> ...snip... XSLT Edi2Xml.exe HL~4~3~8~1 SLN~5.0~~I~1~M4~11.85~~I~SV~0705 Edi To Xml – Slide 16
Translated XML – v.2 <HL> <HL01>4</HL01> <HL02>3</HL02> <HL03>8</HL03> <HL04>1</HL04> <SLN> <SLN01>5.0</SLN01> <SLN02></SLN02> <SLN03>I</SLN03> <SLN04>M4</SLN04> <SLN05>11.85</SLN05> <SLN06></SLN06> <SLN07>I</SLN07> <SLN08>SV</SLN08> <SLN09>0705</SLN09> <GroupLevelCharge> <Charge serviceType='Measured Service‘> $11.85 </Charge> ...snip... XSLT Edi2Xml.exe HL~4~3~8~1 SLN~5.0~~I~1~M4~11.85~~I~SV~0705 Edi To Xml – Slide 17
Agenda • Project Motivation • EDI Basics • Syntactic Conversion: EDI to XML • Semantic Conversion: XML to XML • Validating the Input Data • Storing and Using the Results • System Instrumentation • Q & A Edi To Xml – Slide 18
Validating the Input Data EDI transaction sets contain checksums. • Control codes in matched segments (ISA/IEA, ST/SE, GS/GE) must line up. • Some segment-types (like HL) have global sequence numbers, parent-references, nesting indicators: HL~45~37~5~0 HL~(my id)~(my parent’s id)~(my level)~(I have children) • Optional transaction totals: CTT~3 CTT~(# of preceding IT1, or line-item, segments) Edi To Xml – Slide 19
Validating the Input Data – With XML! 1. Hook into existing EDI validation rules using time-tested XML methods: • DTD • W3C XML Schema Language • Schematron 2. Create sys-admin dashboards. E.g.., express this in XSLT: display in bold-red type: any loops whose instance-id is not (previous-id+1) Edi To Xml – Slide 20
Agenda • Project Motivation • EDI Basics • Syntactic Conversion: EDI to XML • Semantic Conversion: XML to XML • Validating the Input Data • Storing and Using the Results • System Instrumentation • Q & A Edi To Xml – Slide 21
Storing and Using the Results EDI to XML makes great sense. But EDI to XML to RDBMS can be tricky. If the goal is to produce a document (like a detailed invoice) just use XSLT to produce HTML, PDF, etc. In many cases, you should just insert into the RDBMS only the values and labels you care about, along with a unique id pointing to the (saved) XML. Edi To Xml – Slide 22
Agenda • Project Motivation • EDI Basics • Syntactic Conversion: EDI to XML • Semantic Conversion: XML to XML • Validating the Input Data • Storing and Using the Results • System Instrumentation (Demo) • Q & A Edi To Xml – Slide 23
Source Code for the Edi2Xml tool, and bothXSLT and document samples are available at: http://home.comcast.net/~blairschneidermckay/ Q & A Blair Schneider McKay Clareos, Inc. http://www.clareos.com BlairSchneiderMcKay@comcast.net