INNOV-12: Transforming Non‑XML Documents with XML Tools

1. INNOV-12: Transforming Non‑XML Documents with XML Tools Tony Lavinio PrincipalSoftware Architect, Stylus Studio

2. Let’s use Stylus Studio! It’s what all the cool people are using • eWEEK 5th Annual Excellence Awards Finalist • Integrated with: • Saxon 6.5.3 and 8.4 from Saxonica • Xalan-J from the Apache Xerces project • MSXML 3 and 4 • System.XML XSLT from .Net • Sleepycat Berkeley DB XML • Mark Logic XDMS INNOV-12 Transforming Non-XML Documents with XML Tools

3. XSLT 1.0 and 2.0 XQuery XPath 1.0 and 2.0 DTD W3C XML Schema SQL/XML JDBC and ODBC SOAP, WSDL, UDDI OASIS Catalogs Java™ JSP JAXP (formerly TrAX) JAXB HTTP[S] FTP [X]HTML EDIFACT/CEFACT X12 Build on Standards Everything-but-the-kitchen sink INNOV-12 Transforming Non-XML Documents with XML Tools

4. XML or Not XML, Who Cares? Stylus Studio can treat non-XML data as XML • XQuery and XSLT only act on XML • CSV, EDI, etc. are not XML • What can we do? • Batch convert? — No! • On-the-fly conversions • Convert-to-XML feature • Native adapters INNOV-12 Transforming Non-XML Documents with XML Tools

5. Agenda • Convert-to-XML feature • The Architecture • XQuery and XSLT on non-XML • Deploying • Running INNOV-12 Transforming Non-XML Documents with XML Tools

6. Convert-to-XML User Interface Maps flat input data to XML output data Properties Pane Schema Pane XML Output Preview Input Canvas INNOV-12 Transforming Non-XML Documents with XML Tools

7. Convert-to-XML Features • Maps text or binary formats • CSV, TSV, EDI — even xBase • Gobs of encodings • Fixed or variable-length fields • Variety of data types supported • Regular expression matching • Streaming engine supports large input files INNOV-12 Transforming Non-XML Documents with XML Tools

8. Demonstration of Convert-to-XML UI • Let’s build a CSV converter! • Let’s build an EDI converter! (Sample of X12 EDI data) <Demonstration/> INNOV-12 Transforming Non-XML Documents with XML Tools

9. Agenda • Convert-to-XML feature • The Architecture • XQuery and XSLT on non-XML • Deploying • Running INNOV-12 Transforming Non-XML Documents with XML Tools

10. Extensible File System Support Lets us treat any type of resource as a URI • Traditional file systems • file:, ftp: and http[s]: (WebDAV) schemes • Web Service Call Composer • wscc: scheme • SQL/XML for Relational Databases • db: scheme • Sleepycat Berkeley DB XML • dbxml: scheme INNOV-12 Transforming Non-XML Documents with XML Tools

11. The Adapter URL Syntax Two forms: • Convert-to-XML: adapter:map.conv?url • Native XML Adapters: adapter:name:[option=value:…]?url INNOV-12 Transforming Non-XML Documents with XML Tools

12. Implementation What’s under the hood • Convert-to-XML has two parts • User interface • Runtime engine • The runtime engine is just one of many native converters that can be plugged into the Adapter File System. • Designed as a streaming engine for low footprint INNOV-12 Transforming Non-XML Documents with XML Tools

13. Adapter File System Stylus Studio file: http: and https: (WebDAV) *.d db: *.txt*.csv*.html*.xslt*.??? custom file system driver java: user: adapter: CtoXML CSV dBase .d It plugs in – and supports its own plug-ins INNOV-12 Transforming Non-XML Documents with XML Tools

14. Agenda • Convert-to-XML feature • The Architecture • XQuery and XSLT on non-XML • Deploying • Running INNOV-12 Transforming Non-XML Documents with XML Tools

15. A Simple Confluence of Ideas 1 + 1 + 1 > 3 + If we combine the idea that adapters convert non-XML to XML, + With the idea that adapters can be addressed with URLs, + With the idea that XQuery and XSLT work on URL-addressable XML, = We have an integration tool INNOV-12 Transforming Non-XML Documents with XML Tools

16. Design XSLT or XQuery Take the non-XML and manipulate it as XML • Build an XSLT transformation • Input is EDI • Desired output is CSV <Demonstration/> INNOV-12 Transforming Non-XML Documents with XML Tools

17. Agenda • Convert-to-XML feature • The Architecture • XQuery and XSLT on non-XML • Deploying • Running INNOV-12 Transforming Non-XML Documents with XML Tools

18. Assembling the Building Blocks Here’s what we’ve done: • Used the Convert-to-XML map we built earlier for input • Used a CSV file via adapter as the target • Built XSLT from and to representative non-XML documents And here’s what we’re about to do: • Build a Java program to call XSLT • Call the Java program from the 4GL INNOV-12 Transforming Non-XML Documents with XML Tools

19. The Runtime • We use Saxon 6.5.3 or 8.4, or Xalan-J as the XSLT engine, or Saxon 8.4 as the XQuery engine • The Stylus Studio runtime adds the missing pieces, such as our URL resolver and adapter libraries • We’ll even generate the code for you INNOV-12 Transforming Non-XML Documents with XML Tools

20. Calling Our Transform This is just plain standard JAXP code // This is the “before” version… String xslURL = "file:///c:/temp/EXCHANGE.xsl"; String xmlURL = "file:///c:/temp/EXCHANGE.xml"; Source xsl = new StreamSource(new URL(xslURL).openStream(), xslURL); Source xml = new StreamSource(new URL(xmlURL).openStream(), xmlURL); Result out = new StreamResult(System.out); TransformerFactory factory = TransformerFactory.newInstance(); Transformer t = factory.newTransformer(xsl); t.transform(xml, out); INNOV-12 Transforming Non-XML Documents with XML Tools

21. Same, But with Stylus URL Resolver Add two property settings, and use the new factory System.setProperty(STYLUS_ROOTDIR, "."); System.setProperty(STYLUS_APPDATA, "."); String xslURL = "file:///c:/temp/EXCHANGE.xsl"; String xmlURL = "adapter:EXCHANGE.conv?file:///c:/temp/EXCHANGE.edi"; Source xsl = StylusFileFactory.getFactory().resolve(xslURL, null); Source xml = StylusFileFactory.getFactory().resolve(xmlURL, null); Result out = new StreamResult(System.out); TransformerFactory factory = TransformerFactory.newInstance(); Transformer t = factory.newTransformer(xsl); t.transform(xml, out); INNOV-12 Transforming Non-XML Documents with XML Tools

22. URI Resolvers are for Writing Too Non-XML to XML XML to Non-XML Both Ways Since adapters can go either way or go both ways, so can the URI resolver. INNOV-12 Transforming Non-XML Documents with XML Tools

23. Agenda • Convert-to-XML feature • The Architecture • XQuery and XSLT on non-XML • Deploying • Running INNOV-12 Transforming Non-XML Documents with XML Tools

24. Finishing up… • Compile the .java • Move things into the proper locations • Set up the 4GL • Go for it! INNOV-12 Transforming Non-XML Documents with XML Tools

25. Let’s See It All Work Together EDI  XML  XSLT  CSV  4GL = + <Demonstration/> INNOV-12 Transforming Non-XML Documents with XML Tools

26. Resources • Stylus Studio Web Sitehttp://www.stylusstudio.com • Stylus Scoop Newsletterhttp://www.stylusstudio.com/scoop • The W3C XSLT and XPath Standardshttp://www.w3.org/Style/XSL/ • The W3C XQuery Standardhttp://www.w3.org/XML/Query#specs INNOV-12 Transforming Non-XML Documents with XML Tools

27. In Summary • Stylus Studio lets you treat non-XML data sources as XML • Nothing beats XSLT/XQuery for transforming XML • The XSLT/XQuery Mappers make standards-based integration simpler INNOV-12 Transforming Non-XML Documents with XML Tools

28. Questions? INNOV-12 Transforming Non-XML Documents with XML Tools

29. Thank you for your time! INNOV-12 Transforming Non-XML Documents with XML Tools

30. INNOV-12 Transforming Non-XML Documents with XML Tools