1 / 51

National Information Exchange Model Looking Forward IJIS Industry Brief

National Information Exchange Model Looking Forward IJIS Industry Brief. John Wandelt Georgia Tech Research Institute. A Brief History of GJXDM and NIEM. Sep 2002 Work begins on GJXDM Dec 2002 GJXDM beta presented to Global Apr 2003 GJXDM pre-release 3.0.0.0 released

ellis
Download Presentation

National Information Exchange Model Looking Forward IJIS Industry Brief

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. National Information Exchange ModelLooking Forward IJIS Industry Brief John Wandelt Georgia Tech Research Institute

  2. A Brief History of GJXDM and NIEM • Sep 2002 Work begins on GJXDM • Dec 2002 GJXDM beta presented to Global • Apr 2003 GJXDM pre-release 3.0.0.0 released • Jan 2004 GJXDM 3.0 released • May 2004 GJXDM Developer Workshop - Atlanta • Nov 2004 GJXDM 3.0.2 released • Feb 2005 DoJ and DHS sign MOA to build NIEM • Jun 2005 GJXDM User Conference – Atlanta • Sep 2005 GJXDM 3.0.3 • Sep 2006 GJXDM User Conference – San Diego • Oct 2006 NIEM 1.0 (based on GJXDM 3.0.3) • Jun 2007 NIEM 2.0 (merged GJXDM 3.1beta) • Aug 2007 Information Sharing Conference – Chicago • Jan 2009 You are here ... So, what’s happening ???

  3. NIEM Fundamentals • NIEM profiles XML Schema to reduce its complexity. • NIEM extends XML Schema to compensate for its hierarchical nature and representational constraints: • ID / IDREF • Associations and roles • Metadata on metadata • Type Augmentation • Abstract elements and substitution (groups) • Many tradeoffs: • Interoperability vs. Flexibility • Simplicity vs. Precision • Components first vs. IEPD requirements • Flexible + adaptable vs. fixed + standard • Enumeration vs. free text • Elements vs. attributes • Referenced content vs. inline content • Semantics is key.

  4. You Are Smarter Than A Computer! Olnysrmatpoelpe can raedtihs. I cdnuoltblveieetaht I cluodaulacltyuesdnatnrdwaht I was rdanieg. The phaonmnealpweor of the hmuanmnid, aoccdrnig to a rscheearch at CmabrigdeUinervtisy, it deosn'tmttaer in wahtoredr the ltteers in a wrod are, the olnyiprmoatnttihng is taht the frist and lsatltteer be in the rghitpclae. The rset can be a taotlmses and you can sitllraed it wouthit a porbelm. Tihs is bcuseae the huamnmniddeos not raederveylteter by istlef, but the wrod as a wlohe. Amzanig huh? yaeh and I awlyastghuhotslpeling was ipmorantt!

  5. One Year’s Experience with NIEM 2.0 and IEPDs • Components: ~4500 (most types & properties) • Domains: 1 large (Justice); 6 smaller (primarily DHS) • IEPD Clearinghouse (registry for GJXDM & NIEM): - 76 NIEM IEPDs • NIEM IEPD Registry/Repository (niem.gov): - 58 shared NIEM IEPDs from 21 different sources

  6. NIEM Help and Issues Processing Assign ticket # NIEM Config Control Tool Level 1 NCCT NIEM KB Level 2 NIEM Tools Level 3

  7. NIEM Focus Group • Conducted by NCOC • 25-26 June 2008 Wash, DC • 20+ developers and practitioners representing Federal/State government, industry, NTAC, NBAC • Identified 24 strengths for building info exchanges • Identified 51 weaknesses, issues, suggestions: value proposition conformance toolsIEPD development governance contentbest practices architecture trainingchange mgt implementation

  8. NIEM Leadership Conference Themes • Domain Independence • Self Service • Ability to Scale • Quality Assurance and Metrics • Open Interfaces for Tools and Life Cycle Processes • Conformance: Clear Guidance and Tools • Reuse of IEPDs and Components • Missing Content and Domains • Timeline for Releases

  9. High Priority Work Packages • Data Model Maturity Life Cycle (DMMLC) • Version Architecture • Quality Assurance and Release Strategy • Tool Architecture • Minor Release 2.1

  10. … CodeLists Data Model Maturity (DMM) Life Cycle NIEM Domains NIEM NIEM Core NIEMcomponents … … Structures Universal reuse Common IEPDs add / replace G reuse / revise J add / replace Intel EM PS Im IP IEPD Library New domain D3 D2N IT add

  11. Dependencies C1.0 NIEM 1.0 IT Im J Sc In IP EM Geo C2.0 NIEM 2.0 IT Im J Sc In IP EM Geo

  12. Example of Incoherent Schemas uses

  13. Version Architecture Objectives • Domains may publish updates on their own timeline. Publication does not wait for NBAC review. • Domain updates are quickly available for use by IEPDs. Updates may be used by IEPDs without being delayed by updates, synchronization, or harmonization. • Domain updates are incorporated into next NIEM release. • IEPD developers are provided with an updated schema set that is coherent, for increased usability. • IEPDs have the flexibility to use NIEM components as required to satisfy their business requirements - Even previous releases. • Provides a well defined process and schedule for domains to input to the NBAC update, synchronization, harmonization processes for future NIEM releases.

  14. Version Architecture Process

  15. Quality First!

  16. Quality Assurance Layers well-definedsubmissionformat and review process quality release products, tools, documentation Operations (process) Harmonization Inputs to model Output products Data Metrics / stats Validation checks Inspection / review no semantic overlap or duplication complete, correct, consistent

  17. Example Metrics and QA Checks • Data • ISO 11179 naming • Definition quality (semantics) • Descriptive metadata • Code lists: currency, associated text property • NDR conformance • Harmonization • Identification of dependencies • Component name similarity • Duplication and semantic overlap • Namespace cohesion • Depth of type hierarchy (inheritance depth) • Operational • No. of new domains • No. of model changes • No. of impacted IEPDs • No. of extended components in IEPDs • Component usage in IEPDs

  18. NIEM Tool Architecture

  19. Conformance Services

  20. NIEM NDR 1.3

  21. NIEM Conformance • Three types of conformance: • NIEM XML schemas conform to the NIEM NDR. • NIEM XML instances conform by validating to NIEM-conforming schemas (and instance conformance rules specified by the NIEM NDR). • NIEM IEPDs conform to the NIEM IEPD Specification (which requires their XML schemas and instances are NIEM-conforming). • An IEPD conforms to NIEM under the following conditions: • Each XML schema adheres to the NIEM NDR for its conformance class (i.e., subset, extension, exchange, or constraint schema rules). • Each XML sample instance adheres to the NIEM NDR for XML instances. • The IEPD itself adheres to the NIEM IEPD Specification (including required files, packaging, metadata, etc.).  • If an existing NIEM component matches IEPD business semantics, then that component is used by the IEPD (directly or for derived components).  (i.e., IEPD does not unnecessarily duplicate NIEM components) • Each use of a NIEM component by the IEPD (directly or for derived components), is consistent with the component's structural definition and business semantics. (i.e., IEPD preserves semantic and structural consistency) • There are subjective factors in applying these rules that require diligent consideration by the organization(s) developing the IEPD.  Decisions on semantics must be made (or reviewed) by business SMEswith thorough knowledge of the exchange business processes and domain.

  22. What About Systems, Tools, and Databases? • Systems, tools, and databases DO NOT and CANNOT conform to NIEM. • Only XML schemas, XML instances, and IEPDs can conform to NIEM. • For any other entity or artifact, NIEM conformance is undefined. • Internal names for or usage of data within a given system, tool, or database have absolutely no impact on the determination of NIEM conformance. • Conformance is ONLY about the format of payload data encapsulated in XML instances that validate to XML schemas that adhere to the NIEM NDR. • An XML schema that copies, maps to, or uses NIEM names or components without importing NIEM namespaces does NOT conform. • Conforming to NIEM requires that exchange schemas reuse NIEM reference schemas (or subsets) by xsd:import-ing NIEM schemas (or valid subsets) that define NIEM namespaces that compose NIEM releases. • A tool, system, or database may have capabilities that specifically support: • Development of NIEM-conforming IEPDs • Implementation of NIEM-conforming IEPDs • Testing/verifying NIEM-conformance • Generation, sending/receiving, and/or processing of NIEM-conforming exchanges. • Such tools or systems are NIEM-aware or NIEM-supporting (NOT conforming).

  23. test.account@test.com

  24. NIEM Conformance Report Tabs All rules NDR rules IEPD rules

  25. Summary (Auto Checks) 43 of 185 NDR rules auto checked Number of distinctNDR rules checked, passed, failed IEPD Metadata and Catalog result

  26. Summary (Manual Checks) Number of distinctNDR rules not auto checked Number of distinctNDR rules manually verified (by human)

  27. NDR – All Rules (sample) Each of 185 NDR rules listed once with ID number, conformance targets, description, and indication of auto/manual and pass/fail.

  28. NDR – Schemas Lists each schema file checked with indication of conformance

  29. NDR – Rules Auto Failed Filename (of schema) Line No.in file Rulefailed Rationale for failure Type of XML component involved Lists each failure by filename, line #, rule, rationale, type component

  30. NDR – Rules Auto Failed (zoom) Line No.in file Rulefailed Rationale for failure Type of XML component involved

  31. IEPD – Metadata Status Not complete

  32. IEPD – Catalog

  33. Conformance Testing Assistant (ConTesA) • Developed for N-DEx program, although geared towards LEXS in general. • Performs XML Schema validation of LEXS instances, plus any included Structured Payloads. • Performs business rule validations: • NIEM rules (e.g. namespaces and aliases). • LEXS rules (e.g. mandatory elements have content). • Structured Payload specific rules (defined by community that developed Structured Payload schema). • Provides visualization of instance. • Business rule checking available as separate command line tool.

  34. Registry Interfaces

  35. NIEM Minor Release 2.1Features • New DHS domain: CBRN – Radiological Nuclear • New domain: Family Services • DHS Emergency Management will replace current content. • DHS Infrastructure Protection will replace current content. • Updates to DHS Screening, International Trade, Immigration • Update to Geospatial (external standard) • Updates to DoJ Justice (jxdm). • Over 700 missing component definitions will be added. • Cross-domain harmonization will be applied to ... • areas already identified as potential problems. • obvious problems identified in new submissions. • Resolved NCCT issues that can be applied will be integrated.

  36. NIEM Technical References / Documentation • NIEM Releases 1.0 and 2.0 • Reference schemas • Spreadsheet • Model databases (in CSV, XLS, and MDB) • URI pages at http://niem.gov/niem (components, namespaces, xsd’s) • NIEM Introduction (an executive summary) • Concept of Operations (ConOps) • NIEM Terms and Definitions v1.0 (a glossary) • Using Intel Community Info Security Marking (ICISM) w/ NIEM • Specifications: IEPD Requirements, Wantlist xsd, Conformance • Naming and Design Rules (NDR) v1.3 • NIEM Conformance • Techniques for building and extending NIEM XML components • Quality Assurance Strategy & Plan (QASP) • User Guide (Volume I) (Draft) • High-Level Version Architecture (HLVA) • High-Level Tool Architecture (HLTA)

  37. Bridge information systems ________ NIEM Design your XML schemas to be NIEM-conforming 1st Edition Compliments of DoJ and DHS Forward by PMO, NTAC, & NBAC

  38. “ We can’t solve problems by using the same kind of thinking we used when we created them. “ -Albert Einstein

  39. Stovepipe Thinking Must shift thinking from stovepipe to cloud or federation concepts.

  40. Questions Reference: http://niem.gov

  41. Related Projects

  42. Universal Core 2.0 (UCore2) Objectives: Information exchange framework that is interoperable, flexible, applicable, and adoptable by a broad audience. Small core of reusable information objects universally understood and extensible. Information sharing across DoD, DNI, DoJ, and DHS. Impact: Establishes a baseline standard for exchanging information on terrorists and associated activities among U.S. Federal, State, and local agencies. Allows flexibility to extend core objects to satisfy specialized data requirements in local domains (especially, DoD/DNI). Enables interoperability at a semantic level as well as some representational. Description: A joint DoD/DNI/DoJ/DHS project to (1) design a small, universally understood core of information objects and basic relationships, (2) implement in XML Schema, and (3) employ them to exchange and share information across a broad base of stakeholders.

  43. LEISP Exchange Specification (LEXS) SPONSORS: Law Enforcement Information Sharing Program (LEISP) Department of Justice (DoJ) OBJECTIVES: Define and consistently describe units of information to be shared Define interfaces and protocols to provide (publish) as well as search and retrieve such Ensure all agencies can understand a common set of data, without limiting the total data exchanged TASKS: Provide an extensible framework for consistent packaging of information Design for multiple levels of understanding so all agencies can process a common set of data, while allowing communities to add content for their needs without impact to other implementations Utilize NIEM as the baseline data model Provide tools, documentation, and technical support to implementers • TECHNOLOGIES: • eXtensible Markup Language (XML) • Use of NIEM model and NDR • Interoperability • Extensibility • Federated Search • Data Publication

  44. X The “IEPD Consistency” Problem Definition: Two groups independentlydeveloping IEPDs for the same purposemay (nay, will) create incompatible IEPDs. Result: Small-scale interoperability between coordinating partners, but not large-scale interoperability between independent community members (i.e., the ultimatepromise of standards) Mitigations: • Top-down standards: requires “center of mass” • Formal standards: long, arduous process • Consensual standards: hard to gather and govern enough participants to “tip”

  45. LEXS Extensibility

  46. CTISS Federated Registry CTISR NIEM Registry Federated RegistryQueries

  47. September 24, 2008 Global Federated Identity and Privilege Management (GFIPM)Presenter: John Wandelt, GTRI GFIPM Project Manager it.ojp.gov/GFIPM

More Related