1 / 58

Business Unit Empowerment Through Declarative Systems

M. D. Metadata Solutions. Business Unit Empowerment Through Declarative Systems. Why IBM DB2 PureXML™ Rocks! Dan McCreary Dan McCreary & Associates September 11, 2007. DISCLAIMER.

ramiro
Download Presentation

Business Unit Empowerment Through Declarative Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. M D Metadata Solutions Business Unit Empowerment Through Declarative Systems Why IBM DB2 PureXML™ Rocks! Dan McCreary Dan McCreary & Associates September 11, 2007

  2. DISCLAIMER • The following views are my own and do not necessarily reflect the views of the Minnesota Department of Revenue or the State of Minnesota

  3. Presentation Abstract Many non-programmers feel that complex Java and .Net middle-tier systemprevents business units from being able to quickly build and modify simpleweb applications (aka CRUD applications).  DB2 version 9 has powerfulsupport for quickly storing and searching complex XML documents.  Thisfeature combined with storing XML in a web browser using XForms makesmiddle-tier code unnecessary, putting non-programmers in control ofapplications.This presentation will cover a case study that allows a business unit to"draw pictures" to build simple web applications.  These pictures (XMLSchema diagrams) are transformed directly into XForms and stored in DB2version 9.  Rapid searching is provided using XQuery.  Very little (ifany) procedural code is involved in this process.  The presentationfinishes with a set of 10 recommendations that empower business units tobuild and maintain their own applications and leverage the power of acentral DB2 database.This presentation is appropriate for business and data strategists as wellas DBAs and application architects.

  4. Agenda • How did we get here? • Why trees not tables? • Demos • NIEM, XML Schemas, XForms • Costs of translation from XML to OO to tables • XQuery in DB2 • DB2 and your Enterprise Technology Strategy

  5. web Incoming! Has this web thing gone away yet? XML

  6. More New XML Data is Being GeneratedThan Relational Data • Growing at 2X rate of total database market [IDC] • XML is now pervasive in many of organizations • Almost every sector has XML based standards"XML needs its own entry here to reflect its transformation of information … XML isenabling the revolution of Web services and driving a database revolution."-- eWeek "25 More Top Technologies", September 18, 2006

  7. A Happy Partnership XForms DB2 PureXML

  8. Strategic Consequences • Empowerment of business units to build and maintain their own applications with much less involvement of IT staff • Dramatically reduced need for “middleware” IT staff using traditional procedural languages like Java and C# • Evolution of DB2 to a web service data providor and core component of ESB

  9. Did you Know… • That with XForms you can build rich-client web applications without writing a single line of JavaScript? • That IBM DeveloperWorks has almost 300 articles that discuss XForms? • That DB2 version 9 "PureXML" has fantastic support for native XML data types and supports W3C XQuery standards?

  10. How did we get here? • From punch cards… to tables… to trees… to cubes and graphs…

  11. In the Beginning… There were puchcards… …and there were 80 columns… and the widths were fixed… and batches of cards were called "decks“… and they were loaded into tables in “core” memory…

  12. And Codd and Date said… • Let there be relationships… Person Role Project person_id person_id project_id first_name project_id project_nm last_name role_nm description birth_date time_pct status and we had the relational databases… and we had SQL… and we had joins… and we had ER diagrams… and we had PowerBuilder… and all seemed right with the world…until

  13. Sir Berners-Lee Said… • Let there be HTML… <html> <head> <title>Dan’s Homepage</title> </head> <body> <h1>Welcome!</title> <p>Welcome to my web site!</p> <a href="about.htm">About Me</a> </body> </html> and we had URIs… and we had HTTP… and we had we links and resources… and we had trees (not tables)… and we had search…until

  14. Customers Said.. • We wanted to access our tables…on the web…in HTML! html ? head body h1 p a and there were objects… and there was Java and JSP … and there was ASP.Net and there were models, views, controllers… and there was conversion from tables to objects to trees and trees to objects to tables… and there were rooms and rooms of programmers…

  15. And there Was Chaos… HTML Forms html Objects JSP/ASP head body Hibernate Serialization h1 p a 100+ Frameworks OR Mapping Struts JavaScript XML JDBC SQL .Net AJAX Sessions Stored Procedures And the number of languages you need to learn and the headcount of the IT departmentwent up…and up RDBMS

  16. The Translation “Pain Chain” • From web forms to objects…to SQL inserts…to selects…to objects and back to web forms • Many format translations… Name: Street: City: Zip: Web Forms Objects RDBMS

  17. Person FirstName LastName Roles Projects Project And the W3C Said… • Let their be simplicity…what if we…allowed you to XML in the browser…and database html Database head body model input label

  18. A Single Line of SQL INSERT into MY_TABLE values(XMLPARSE(document ' <MyDocument> … </MyDocument> ')) The equivalent SQL for the eCRV project would have required “shredding”the XML document into over 25 tables (i.e. 25 separate INSERT statements).

  19. Enter the Metadata Registry… • A metadata registry is a central location in the enterprise that the semantics or meaning of data elements is captured and maintained <DataElement> <Namespace>http://purl.org/dc/elements/1.1</Namespace> <DataElementName>ResourceTitleText</DataElementName> <Label>Title</Label> <Definition>The name given to the resource, usually by the Creator or Publisher.</Definition> </DataElement> Begin with the end in mind…semantic precision.

  20. Demos • NIEM Subset Generator http://niem.gtri.gatech.edu/niemtools/ssgt • Certificate of Real Estate Value • http://proptax.mdor.state.mn.us:8088/exist/rest//db/crv/forms/submit/index.xhtml • Contacts Demo • http://proptax.mdor.state.mn.us:8088/exist/rest/db/contacts/index.xhtml

  21. XML Schema (XMLSpy Diagram) Solid linemeans required Data Element Sequence of items Dashed linemeans optional Cardinality Definition Graphical notation that most non-programmers can learn in 20 minutes.

  22. XForms User Interface (Generated) Use XSL to transform XML Schema directly into Xforms Use the metadata registry to extract element-specific rules Separate semantics from constraints

  23. Metadata Driven Apps Data Flow NIEM MDR wantlist.xml Local MDRs xforms-global.css field-widths.css NIEM Sub Schemas .xsd instance.xml patterns.xml read-only.xml screen-labels.xml import My-Form.xhtml import DocumentConstraint Schema .xsd import niem2xforms.xsl This is a simplified version of the actual workflow.

  24. How About Reporting on XML data? • OK… it is easy to get XML into DB2.. …but how about reporting? How do I do “joins” on XML? …How do I update XML documents? …Can I mix SQL and XQuery?

  25. DB2 Viper Architecture • Many benefits of mix and match SQL/Xquery • Use the right tool for the right job

  26. XQuery • W3C Recommendation 23 January 2007 • Created by experts from SQL industry • IBM, Microsoft, Oracle • IBM played a leadership role • XQuery is to XML as the SELECT statement is to SQL

  27. XQuery Structure • FLWOR expression (pronounced “flower”) • For – select the sequence of nodes in the tree • Let – temporary variables and lists • Where – restrict results • Order – order results • Return – return the data embedded in a tree

  28. Sample XQuery for $d in db2-fn:xmlcolumn(“depts.depts")//deptno let $e := db2-fn:xmlcolumn(“emps.emps")//employee[deptno = $d] wherecount($e) >= 10 order by avg($e/salary) descending return <big-dept> { $d, <headcount>{count($e)}</headcount>, <avgsal>{avg($e/salary)}</avgsal> } </big-dept> Returns a list of all big departments (over 10 employees), their headcount and average salary directly in XML format

  29. XSLT and XQuery • Functionally similar languages developed by the W3C • Working together to ensure a common approach where appropriate. • They share the same • data model • type system(XML Schema data types) • function library • Both us XPath 2.0 as a sublanguage

  30. What I like About XQuery • XPath expressions are very short but can still easy to read (c.f. Perl) • Consistent with XSL, XForms, XML Schema and Schematron (XML rules) • You can extend XQuery using stored procedures • Recursion simplifies many tasks

  31. But Can I… • Do joins between XML documents…yes • Extend XQuery…yes • Create “functions” with XQuery…yes • Integrate Java functions…yes

  32. Joins in XQuery declare variable $firstName external;<videosfeaturing="{$firstName}">{for $v in db2-fn:xmlcolumn(‘videos’)//video,      $a in db2-fn:xmlcolumn(‘actors’)//actors/actorwhere ends-with($a, $firstName)and $v/actorRef = $a/@idorder by $v/yearreturn      <videoyear="{$v/year}">         {$v/title}      </video>}</videos> List of videos in one file List of actors in another Return a list of all videos That have an actor with a given first name.

  33. XQuery Update • Adds XQuery expressions to: • Insert • Update • Delete XML documents • Still in W3C Working Draft • Last call for Comments is Oct. 2007 • Should be full recommendation by 2008 • Available in DB2 9.1 as a stored procedure • Additional support in DB2 9.5

  34. Stored Procedures For Update CREATE PROCEDURE ECRVADMIN.INCREMENT_COUNTY_CRVID (IN COUNTY VARCHAR(255), OUT SUCESS INT) LANGUAGE SQL BEGIN SET ARG1 = CONCAT( '<updates><update action="compute" col="1" path="//County[Code=&quot;' COUNTY &quot;]/NextID/text()">) CPMCALL DB2XMLFUNCTIONS.XMLUPDATE( ARG1, ‘ select COUNTY from ECRVADMIN.COUNTY where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?); CALL DB2XMLFUNCTIONS.XMLUPDATE( '<updates> <update action="compute" col="1" path="//County[Code=&quot; &quot;]/NextID/text()"> ?+1 </update> </updates>', 'select COUNTY from ECRVADMIN.COUNTY where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?); CALL DB2XMLFUNCTIONS.XMLUPDATE( '<updates> <update action="compute" col="1" path="//County[Code=&quot;COUNTY&quot;]/NextID/text()"> ?+1 </update> </updates>', 'select COUNTY from ECRVADMIN.COUNTY where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?); END

  35. Sidebar: The Semantic Web Subject Predicate Object Node • Note that the semantic web infrastructure is based on graphs, not trees • Each node in the graph can be a URL • External web sites can be used to create “joins” based on a URL Node Node Node Node Node

  36. Strategic Impact • Consider this: • 50% of Applications are simple CRUD Applications • Create, Read, Update, Delete • Complex business logic can be stored in an XML-based rules engine such as Schematron • How will this transform an organization? • How will it allow business units to build and maintain IT applications? • How will it allow the business to be responsive to custom needs?

  37. Graphical Requirements and Specifications SME/BA IT Staff Change Where the Line is Drawn • Shorten the “distance” between the business unit and the IT staff Requirements Requirements BAs SME Developers vs.

  38. XForms XQuery XPath XSLT XML Schema Requirements Schematron Many Small vs. One Large It is easier to build a GUI on a small, precise language than a single large language

  39. Data Stewardship/Governance • Importance of the business units role in defining data elements semantics “on-the-wire” • How to write and maintain enterprise definitions • Concise • Precise • Distinct • Non-circular • Unencumbered

  40. Security and Performance • Security and access control based on URL patterns • A middle tier still needed to cache web pages that have similar URLs • Deep REST considerations (don’t regnerate a pages if you don’t have to) • Stateless web server farms can still be used

  41. Modules and Quality Assurance • XQuery “Modules” allow encapsulation of functions to manipulate complex XML • Need better unit testing tools for XQuery (similar to JUnit) • Need support for regression testing frameworks (similar to CruiseControl)

  42. Procedural code (Java, JavaScript, VB, C#, C++) Declarative code (XHTML, CSS, XSLT, XQuery, XForms) Parker Projection RelativeCode Base 100% Time Source: Jason Parker, Minnesota Department of Revenue, November 2006

  43. 10 Business Empowerment Steps • Make data stewardship a business unit imperative • Start by putting a simple glossary of terms and their definitions in a database • Build a simple metadata registry based on a simple taxonomy • Build a metadata "shopping cart" • Build exchange documents using XML Schemas that import your metadata shopping lists • Transform XML Schemas into XForms • Put your XML data in DB2 XML data types (no shredding!) • Use XQuery for reports directly into XML and HTML • Express level-two business rules by selecting XPath expressions • Build XML-centric ESB with DB2 databases as sources

  44. Next Steps • Proof of Technology (POT) – hands on labs • Get the evaluation version…start with some small XForms and XQueries • Create a pilot project – goal – small middle tier • Pick a "friendly" business unit • Find a medium sized problem • Train staff with XPath-centric tools • Roll out to the enterprise in stages

More Related