NIEM, CAM and the 8 “D’s”

1. National Information Exchange Model XML Exchange Development Deploy Requirements NIEM Model Data Test NIEM, CAM and the 8 “D’s” Build Exchange Generate Dictionary David Webber - Public Sector NIEM Team, December 2011

2. The following is intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. DisclaimerNotice

3. Why CAM and NIEM? *CAM editor toolkit – http://www.cameditor.org and OASIS Content Assembly Mechanism standard – http://docs.oasis-open.org/cam “Can you give us a brief overview of how you expect IT professionals and line of business managers to use CAM*?” – Van Ristau, CTO DLT Inc.

4. Response Line of business managers can use CAM to drive better alignment to NIEM and faster more consistent delivery of NIEM information exchanges enabling projects to be performed more timely and predictably CAM provides tools that allow business analysts to work collaboratively with software developers to build exchanges Teams use CAM to perform the 8 “D’s” tasks when implementing NIEM exchanges CAM automates the delivery of key XML artifacts for NIEM exchange services CAM can be deployed using the CAMV engine

5. OASIS CAM standard and approach CAM is about content assembly for information exchanges using a visual XML paradigm with the components managed in dictionary collections NIEM provides the basis for domain dictionaries and the naming and design rules for creation of components NIEM aligns well with CAM because they share a common heritage in core component standards work CAM is designed for standards based business exchange development and simplifying that process CAM works for both top down new exchanges or bottom up customizing and tailoring of existing exchanges

6. Introducing the 8 “D”s Tasks in implementing NIEM Design Develop Deploy Document Dictionaries Discovery Differentiate Diagnose

7. The 8 “D”s and NIEM NIEM IEPD Process Repeatable, Reusable Process (Exchange Specification Lifecycle) *IEPD - Information Exchange Package Documentation Design Develop Deploy Document Dictionaries Discovery Differentiate Diagnose

8. Exchange Delivery Lifecycle Dictionaries Discovery Requirements Updates Design Drag and Drop Visual Designer Diagnose Develop XML Samples Exchange Templates Differentiate Production Results XSD Schema Deploy Document NIEM IEPD Reports Validated Templates / Schema XMI / UML Models Documentation

9. CAM: Top Down Exchange Assembly Components Canonical Dictionary Collections Canonical XML Components Dictionary 1 XML User Inserts Navigation and Query Tools Relationship Lookups Component Associations and Couplings 2 Domains Reuse Library Exchange Templates and Rules 3 Artefact relationships Catalog Template Presentation Automatic Rendering Tools W3C Schema and Model Representations 4 Collaboration Services XML Schema Components Models Delivery Control, Messaging, Security Deployment Environments and Middleware 5 Testing Workbench, Rules Engine, Data Samples, Integration ETL Implementation Artifacts and Examples 6

10. Design Create information exchange conceptual work flows including the information exchanges needed to match business process needs Determine conceptual information exchanges Understand security and information privacy needs Determine the information components for each exchange Organize the components in each exchange structure to an overall hierarchy to align with the business process needs CAM provides a visual editor for prototyping of exchange structures along with Mind map modelling to allow conceptual reviews

11. Discovery • CAM provides a variety of tools to assist in discovery of existing components from dictionary collections • Automatic comparison tool between exchange and dictionaries – creates mapping spreadsheet • Search tools within dictionary interface • Visual dictionary structure viewer • Mind map models of dictionary components to allow further visual inspection and analysis

12. Develop • Using the CAM exchange structure outlines complete the actual exchange details • Select actual components either from existing NIEM dictionaries or from the enterprise dictionaries • Create new components or extend existing components to meet additional design needs • Add validation rules for structure and content as needed by the business process • Build working exchanges using actual simulated data with realistic XML instances • Test each exchange to verify validation and operation • Integrate the information exchange handling with the business processes and applications

13. Component Definitions Component Definitions Visual Designer with Dictionary Collection Collection Domain dictionary Industry dictionary 1 Exchange Designer 2 Search Tools Drag n’ Drop Tool Insert Dictionary Parent Components 3 4 5 Completed Exchange Template

14. SAR conceptual components • SAR v1.5 components • NIEM core dictionary • LEXS 3.1.4 dictionary LEXS components referenced dictionaries NIEM core components XML XML XML New structure components based on NIEM + SAR + new Definitions stored as syntax neutral canonical XML Example - Suspicious Activity Report V2.0 Dictionary Collection DRAFT Namespaces of dictionary components

15. Document • CAM generates NIEM IEPD documentation artifacts including • Business component usage report with rules and definitions • Code list details and content checks • UML physical models • Mind map model of exchange structure and components • Comparison mapping spreadsheets to NIEM dictionaries and components • NIEM “want list” XML for use with NIEM subset generator tool • Example XML instances

16. Differentiate • This step includes building in deployment specific details and rules and usage policy determinations • Add additional XPath rules for local integration needs • Constrain code lists to local use • Limit and restrict content based on policy and role of exchange partners • Contextually exclude structure components based on rules • Create other integration artifacts for middleware such as policy control, partner certificates and security configuration • Can configure these aspects through the CAM template and using middleware tools

17. Deploy and Diagnose CAM Editor resources site: http://www.cameditor.org • Once structure information exchange is complete need to test and verify it by generating realistic XML examples • Validate those against the exchange template • Share working examples with exchange partners • Share documentation (IEPD) • Deploy with CAMV and distribute test suite module • CAMV is Spring aware and supports Java API • Tutorial and examples available

18. CAM Validation Framework

19. CAMV XSLT CAMV / ANT Test Suites Environment CAM template Reports CAMV Control Script XML XML Test Instances ANT 2 XML Validation Results 1 Examples and instructions: http://www.cameditor.org/#CAMV_Testing

20. Dictionaries CAM Toolkit provides suite of tools for dictionary handling • Sets of NIEM domain dictionaries available • Dictionaries can be grouped into collections • Tools allow comparisons with existing dictionaries for reuse analysis and mapping spreadsheet generation • New dictionaries can be created by • Harvesting existing schema collections • Export and generation from Enterprise Data Models (ERwin) • Automatic Renaming tool can be used to generate NIEM consistent dictionary components for a domain • Naming and Design evaluation tool provides alignment to NIEM and interoperability reporting and spell checks • Dictionaries can be viewed as spreadsheets or models

21. Developing domain dictionaries (EIEM) CAM Toolkit for EIEM generation *EIEM – Enterprise Information Exchange Model • Allows domains to manage their components libraries • Provides consistency for project development teams • Sets of NIEM consistent XML exchange components • Aligned to enterprise data stores • Optimized for reuse and interoperability • Save time and effort across the enterprise • Perennial question for developers – when should I use NIEM components, and when our own local ones? • Are there components already available for that purpose? • Provide formal mechanisms and procedures to share components and collaborate through project lifecycle • Provide external parties consistent data views

22. CAM Toolkit capabilities review

23. Reviewing NIEM technologies Summary and review

24. Summary of NIEM and CAM Technologies • Rapid NIEM solution development – IEPDs • NIEM and CAM editor toolkit • Ability to use rules, code lists and SQL lookups • Dictionaries and Exchange development • Simplifying and accelerating effective exchanges • NIEM – canonical component management • Tools for building EIEM dictionaries • Drag and Drop designer using dictionary collections • Deployment • CAMV validation XML framework and test suite toolkit

25. Generating IEPD artifacts NDR evaluation, NIEM mapping, “want list” + subset, Exchange Schema, models of the structure components, XML instances, business rules documentation

26. IEPD Generation Steps Structure Rules CAM Template Documentation EXCHANGE TEMPLATE Toolkit for IEPD artifacts generation • Run template Evaluation Report • Compare to NIEM dictionary - create mapping spreadsheet - create wantlist.xml - NIEM online SSGT uploading + subset schema generation Generate business rules report • Generate exchange XSD schema • Generate XML test instance(s) 1 2 Suite of menu options and tools in desktop CAM toolkit editor (each run custom xslt scripts on the exchange CAM template xml) 3 4 5

27. NDR Evaluation Report • Provides scoring, alerts, warnings and potential issues including: • Naming and Definition Conventions • Naming / Restriction consistency checks • Interoperability Enablers/Inhibitors checks • Rules integrity and duplicates • Statistics on exchange size

28. Evaluation Report (NDR) example Part of an example report for LEXS getDataItemRequest template displayed in toolkit HTML viewer

29. Compare to NIEM dictionary • References NIEM dictionary of names and properties currently in NIEM 2.1 or 2.0 releases • Matches on physical names • Reports mapping details • Compatible with Microsoft Excel • Report can be used to do spell checking (in Excel) • Generates wantlist.xml file • Wantlist can be uploaded to SSGT and then subset schema generated (http://niem.gtri.gatech.edu/niemtools/ssgt/SSGT-Options.iepd )

30. NIEM cross-reference spreadsheet Formatted view in Microsoft Excel of import of cross-reference report details (from generated XML file)

31. NIEM online SSGT and wantlist.xml Options menu and Load Wantlist option with resulting load of subset details

32. Documentation and Schema • Documentation: • Create HTML report of exchange schema details and associated content and business rules • Report layout and content designed to be reviewed by business analysts • Schema: • Generate XSD schema for exchange • Customizable IEPD folder layout management by namespace for extension, subset and exchange schema components • Writes XSD schema in syntax that is clear, simple and compatible with deployment tooling environments

33. Business Rules Documentation Part of the example rules for LEXS getDataItemRequest template displayed in toolkit HTML viewer

34. Export Exchange to XSD Schema Export Template to Exchange XSD: Completed Exchange Structure Set Exchange Options Complete set of NIEM exchange schemas generated

35. Exchange Schema Generated Each namespace file is import for those specific type definitions Set of XSD files with filename and namespace suffix Reviewing XSD results in a schema editor tool

36. Exchange Model Generated

37. Examples Generation • Support for software development testing process • Designed to allow creation of concrete realistic examples not just random value based • Hinting system allows insertion of actual test system values into XML examples • Can create both valid and invalid examples to support unit testing of application software • Exclude capability allows generator to create examples that contain only a portion of the entire exchange • Control over random seed value used allows re-generation of identical test cases

38. XML example generation wizard View of CAM toolkit with LEXS getDataItemRequest and dialogue for XML test example generator tool

39. Running validation rules tests • Built-in validation engine allows testing of XML instances against actual exchange rules. • Critical to ensure that the exchange validates actual live production example scenarios correctly • Allows deployed solution to match IEPD exchange details • Errors can be reviewed interactively in exchange visual interface • Post-processing of validation results allows unit regression tests to be created with reporting of errors, warnings and information level notes

40. Run Exchange Template Pick XML test case to validate Run validation Review results in visual editor

41. Formal IEPD Packaging Details

42. Q & A • Resource Center link • http://www.oracle.com/goto/niem Download the CAM editor toolkit from: http://www.cameditor.org

43. Reference Materials References and Links

44. Links and References • SUPPORTING NIEM MATERIALS - • NIEM Naming and Design Rules (NDR) 1.3 • http://www.niem.gov/pdf/NIEM-NDR-1-3.pdf • NIEM Practical Implementer’s Course (online) • http://www.niem.gov/elearning.php • NIEM Course Modules (17 free PDFs) • http://www.niem.gov/course.php • Techniques for Building and Extending NIEM XML Components • http://www.niem.gov/Techniques_for_Building_and_Extending_NIEM.pdf