1 / 63

NIEM Team, Oracle Public Sector

XML Validation Test Suites with CAMV. Exchange Development. Deploy. Requirements. CAMV. Mo del Data. Test. NIEM Team, Oracle Public Sector. Build Exchange. Generate Dictionary. Technology Introduction – September 2011.

dong
Download Presentation

NIEM Team, Oracle Public Sector

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. XML Validation Test Suites with CAMV Exchange Development Deploy Requirements CAMV Model Data Test NIEM Team, Oracle Public Sector Build Exchange Generate Dictionary Technology Introduction – September 2011

  2. The following is not intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. DisclaimerNotice

  3. Test Suites Introduction A key need in working with XML information exchanges, is developing examples and matching templates including rules, context, and roles for correctly managing information exchange content between partners. The W3C Schema by themselves are not sufficient. Learn how to use open source tooling, XPath rules, and the OASIS CAM templates standard to create automated testing tools. Integrating SQL table lookup validation rules The CAMV validation engine can also be used in production environments to manage live information exchanges.

  4. Contents * National Information Exchange Model (NIEM) – see http://www.niem.gov • Part 1 • The XML Validation Framework • XPath Rules Techniques • Code lists and SQL lookups • Rules use cases • Part 2 • Example CAM templates • CAMV validation examples • NIEM example test suite • Elections example test suite • SQL table database lookups • Running validations from visual CAM Editor • Results handling techniques • Summary

  5. Exchange Development Deploy Requirements CAMV Model Data Test XML Validation Framework Build Exchange Generate Dictionary Architecture and Deployment

  6. XML Validation Framework

  7. CAMV Engine

  8. CAMV Validation Engine • Standalone XML validator using XPath rules • Context aware and declarative rules approach • Integrated with CAM editor menus for instant testing • External code list and SQL table lookup support • Callable via Spring framework or Java API; thread safe and server container deployable • Configurable error levels – error, warning • Java error results object returned • Embedded validation results attributes in XML results

  9. CAMV CAM Editor XSLT CAM Editor / CAMV Environment Single XML Test Instance Reports XML Validation Result CAM Editor CAM template

  10. CAMV XSLT CAMV / ANT Script Environment CAM template(s) Reports CAMV Control Script XML XML Test Instances ANT 2 XML Validation Results 1

  11. CAMV Test Control Script XML (single) Singleton Test Case Example <camvTestSuite> <testCase> <Example-Test-v19-3> <cxxfile>Test-Template-v19.cxx</cxxfile> <xmlfile>Test-Instance-v19-3.xml</xmlfile> <verbose>true</verbose> <inlineErrors>true</inlineErrors> <parameters> <abc>Param1</abc> <cde>Param2</cde> </parameters> </Example-Test-v19-3> </testCase> </camvTestSuite> Test case name Validation Template Test XML instance Optional items

  12. CAMV Test Control Script XML (multi-instance) Multi-Test Folder Example Test case name <camvTestSuite> <testCase> <Example-Test-With-Folder> <cxxfile>../samples/templates/EML-330-list-v7.cxx</cxxfile> <testdir>../samples/XMLsamples/UOCAVA/330</testdir> <verbose>true</verbose> <inlineErrors>true</inlineErrors> <parameters> <abc>Param1</abc> <cde>Param2</cde> </parameters> </Example-Test-With-Folder> </testCase> </camvTestSuite> Validation Template Test XML instances folder Optional items

  13. Linux GTK ANT Configuration • Linux distributions (e.g. Ubuntu) has ANT pre-installed in /usr/share/ant/lib • Download Ant-contrib from • http://ant-contrib.sourceforge.net/ • Copy ant-contrib jar file to /lib directory inside ANT installation

  14. Mac OS ANT Configuration • Mac OS X 1.7+ “Lion” comes pre-installed - use command : ant –version and it will self-configure • Mac OS X 1.6 and earlier – need to install “Xcode” development software first • Download Ant-contrib from • http://ant-contrib.sourceforge.net/ • Copy ant-contrib jar file to /lib directory inside ANT installation

  15. ANT – Windows configuration • Download ANT from • http://ant.apache.org • Download Ant-contrib from • http://ant-contrib.sourceforge.net/ • Copy ant-contrib jar file to /lib directory inside ANT installation • Set the ANT_HOME environment variable to the directory where you installed ANT • Add %ANT_HOME%/bin folder to executable path

  16. Windows System Path Settings 1 2 3

  17. Running CAMV with ANT • Go to CAMV location on Sourceforge • http://www.cameditor.org/#CAMV_Testing • Download CAMV release JAR and Test example ZIP • Unpack test example from ZIP and put CAMV jar into folder • Go to the command line (run cmd) • Switch to folder where CAMV test suite is located • ANT runAll • Output is automatically placed in folder called CAMVTestResultslocated within the test instance location folder(s)

  18. Reviewing Validation Results HTML

  19. Exchange Development Deploy Requirements CAMV Model Data Test XPath Rules Techniques Build Exchange Generate Dictionary Quick Syntax Primer Controlling Scope, Context, Evaluation Dynamic Structure Components Useful XPath Resources

  20. XPath Syntax Primer • XPath is a simple assertion language where expressions reference components hierarchically within an XML instance structure – e.g. • /animals/canines/domestic/dogs/poodle[1] • /animals/canines/domestic/* • CAMV uses the XPath v2.0 Java library • In CAM templates XPath is used extensively to provide rules and content details for XML structure components

  21. XPath Functions

  22. CAM XPath Extensions (most used)

  23. CAM templates and XPath expressions Within a CAM template XPath expressions are found in the <as:BusinessUseContext> section of the template.

  24. Exchange Development Deploy Requirements CAMV Model Data Test Rules Use Cases Build Exchange Generate Dictionary Typical Business Needs Quick Examples

  25. Standard Pattern for Business Rules Claim – An English statement of a business rule that conveys its meaning.Example: A Chapter MUST have a Title. Test – A formal statement of a business rule that can be parsed and processed by machine.Example: <Assert test="count(para) &gt;=1"> Diagnostic – An English statement that identifies and explains what happened or was found when a Test fails.Example: Chapter "X" does not have a Title. Outcome – Error, Warning, Pass

  26. Controlling Scope and Context Condition controls if and when the content rule action applies Condition can control structure as well; optional, mandatory, exclude Condition can reference different component(s) for contextual rule actions

  27. Some more examples Business Rule: Items with color attributes don't need a tray number—the color is sufficient for picking the item: <as:constraintcondition="exists(//Item/@color)" action="makeOptional(//Item/TrayNumber)" /> Business Rule: If order weight exceeds 25 kg, purchase order must specify a freight carrier: <as:constraintcondition="//Item/@weight > 25" action="makeMandatory(//Item/FreightHandler)"> Using XPath axis referencing (when things are not adjacent in hierarchy): <as:constraintcondition="exists(ancestor::Item/@color)" action="makeOptional(//Item/TrayNumber)" /> * Excerpted from Michael Sorens DEVX CAM article examples

  28. Useful XPath Resources • Michael Sorens article on using CAM from DEVX.com • http://www.devx.com/xml/Article/41150 • Martin Roberts – tutorial on writing CAM rules • http://merryflame.co.uk/jcam/site/tutorials/CAMTemplateBasics.htm • XPath tutorial resources • Search on XPath tutorial • Books available • Tools – XPath expression constructors

  29. Exchange Development Deploy Requirements CAMV Model Data Test Code Lists and SQL Lookups Build Exchange Generate Dictionary Introduction to CAM code lists Advantages Tools and Hints

  30. Code lists • CAM lookup() function allows referencing to external XML lists of values • Can replace use of restrictValues() static lists • Database - CAM lookup() function can reference SQL table for dynamic value checking • Where to get code lists? • Import XSD enumeration schema into CAM lookup lists • Convert UBL Genericode into CAM lookup lists • Dump from database into lookup list XML format • Direct via SQL table lookup

  31. CAM Code list XML format Code list name Code value Textual value

  32. Combining Rules and Code lists • Real power of code lists and CAM – ability to select code list values based on XPath rule criteria to handle different partner configurations, versioning and cross-table validations • e.g. Ford / Mustang | Chevy / Corvette • Message version id = 1.6 – use code list XYZ-1-6

  33. Code lists examples Load lookup list XML into CAM editor Assign lookup() rule to XML component Using SQL lookups – need to define connectivity to database and the SQL query to perform – we will do this part in the next section

  34. Code Lists Summary Lookups can be to external XML file instances Simple XML format for optimized performance and readability Dynamic runtime control via XPath rules to select lists to be applied Lookup can reference SQL statement for database content checking

  35. Exchange Development Deploy Requirements CAMV Model Data Test Part 2 Build Exchange Generate Dictionary Example CAM templates Running CAMV validations Results handling techniques Summary

  36. CAM templates Here we introduce CAM templates and understanding their component sections Template is abstraction layer over schema structure and rules Template can be automatically built from existing XSD schema Add additional XPath validation rules Compiled templates provide the validation instructions for the CAMV validation engine Using CAMV validations and handling results

  37. CAM Templates Overview • Header declares properties and parameters for the CAM process to reference • AssemblyStructure captures the business information exchange structures • BusinessUseContextprovides the rules to apply during validation • Extensionpoints to the annotations and external lookup tables Consists of 4 functional sections:

  38. Exchange Development Deploy Requirements CAMV Model Data Test Example CAM templates Build Exchange Generate Dictionary Examples Reviewed Quick walk through of important details

  39. Available Test Suite Examples Experimental SAR Examples OASIS EML V7 – UOCAVA Testing • Set of test examples for Election Management: • Both individual test cases and folder based multiple examples • Each test uses different template and validation scenario * SAR – Suspicious Activity Report NIEM exchange * EML – Election Markup Language Standard • Two test example templates and XML instances: • Baseline SAR with just structure and content usage rules, schema parlance “cardinality, facets and enumerations” • Baseline SAR advanced – logical business rules added to check consistency of information and structure components • Each test set contains examples that pass and those that fail depending on the template validation applied

  40. Experimental SAR Overview • Areas of Interest: • Drugs • Fraud • Gangs • Internet • Pornography • Property • Community • Weapons/Chemicals • Prescriptions • Travel/Immigration • Terrorism * SAR – Suspicious Activity Report

  41. Visual Template Rules Wizard Tools

  42. Exchange Development Deploy Requirements CAMV Model Data Test Running CAMV Validation Build Exchange Generate Dictionary Illustrative examples – pass, fail, warnings Default structure handling Data type and Content checking Cross-field validation rules SQL database lookup

  43. Example Advanced Rules Business Rule: Providing warning if PurgeDate is earlier than PurgeReviewDate <as:constraintcondition="//sbl:PrivacyDetails/sbl:PrivacyPurgeDate < //sbl:PrivacyDetails/sbl:PrivacyPurgeReviewDate"action="printMessage(//sbl:PrivacyDetails/sbl:PrivacyPurgeReviewDate,Warning: Review date not before purge date)"><as:annotation/></as:constraint> Business Rule: Location must contain an address or a highway or coordinate set <as:constraintcondition="not(./nc:LocationAddress) and(not(./nc:LocationHighway)) and(not(./nc:LocationTwoDimensionalGeographicCoordinate))"action="printMessage(//j:TargetLocation/sbl:Location,Empty Location information)"><as:annotation><as:documentationtype="documentation">Example of advanced rule</as:documentation></as:annotation></as:constraint>

  44. Default Structure Handling By default all structure components are required and string content Content Masks Structure control

  45. Running CAMV details Batch Command Line Execution Window Viewing Same Template in Visual Editor • From within the Test Suite folder on the command line execute > Ant runALL

  46. Sample SAR Validation Results HTML

  47. Installing and using SQL lookup example Contains test sample and template DB lookup sample install folder contents Run start and stop database scripts Simply download and unpack the ZIP archive and drop the DatasourceExample package into the samples Test Suite folder Then add the camv-test-DbList.xml to the testsuites folder

  48. SQL DB configuration details Modify to match your local directory structure camv-test-DbList.xml contains data source reference and connection configuration Modify inside dbProps.props file to match your local directory structure

  49. Sample SQLDB Test Run • Start the database instance (slide 47) • From within the Test Suite folder on the command line execute > Ant runALL

  50. DB lookup test results rendered as HTML SQL DB lookup result

More Related