1 / 42

DoD Metadata Registry

DoD Metadata Registry. Presentation to Federal Data Architecture Subcommittee. Dr Glenda Hayes ghayes@mitre.org MITRE/DISA PEO-GES 9 Oct 2008. Agenda. Net-Centricity DoD Net-Centric Data Strategy What is Metadata? Structural and Semantic Metadata Discovery Metadata

meagan
Download Presentation

DoD Metadata Registry

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DoD Metadata Registry Presentation to Federal Data Architecture Subcommittee Dr Glenda Hayes ghayes@mitre.org MITRE/DISA PEO-GES 9 Oct 2008

  2. Agenda • Net-Centricity • DoD Net-Centric Data Strategy • What is Metadata? • Structural and Semantic Metadata • Discovery Metadata • DoD Metadata Registry and Clearinghouse (DoD MDR)

  3. Net-Centric Tenets DoD Net-Centric Data Strategy Information Assurance Strategy Core Enterprise Services (NCES) Global Connectivity (Transformational Communications) Recipe for Agility

  4. DoD Net-Centric Data Strategy The DoD Net-Centric Data Strategy, issued by the DoD CIO May 9, 2003, is the plan to make the Department’s information resources: Is an information resource discoverable by most users? Visible Is it available on the network, and are tools readily available to use it? Accessible Can it be intelligibly used? Are the semantics well documented? Understandable Are the source, security level and access controls of the data available to users? Trusted Can it be combined or compared with other information? Can it be mediated? Interoperable Is the data what users need? Are robust user feedback mechanisms in place to improve it? Responsive CC/S/As must institutionalize processes to accomplish these goals 4

  5. Quiz: Which of these is Metadata?           • An XML Schema? A WSDL? • A database structure? • A picklist? • An index card in the library? • MS-Office product’s properties? • Email classification and release conditions? • Notes on the back of a photo? • iPod playlist? • Date and reviewer for a taxonomy? • Product ratings & comments on Amazon?

  6. Metadata Supports NCDS Goals Title: ~~~~~~~~~~ Description: ~~~~ ~~~ ~~~ Author: ~~~ ~~ ~~~~~~ Subject: ~~~ ~~~ ~~~~ Analogy book Information Resource = index card Resource Metadata = template, dictionary, thesaurus Structural & Semantic Metadata =

  7. Mission Provide tools, techniques, and performance standards to enable the DoD Data Strategy Support DISA DoD Data Strategy compliance effort Products & Services Chair DoD Metadata Working Group Provide NCES Metadata Services support Oversee operation of the DoD Metadata Registry Develop and promote Data Strategy Enablers for Communities of Interest (COIs) Provide Data Strategy input to NCES Acquisition Milestone Documents Support NCES Product Branch, Testing & Evaluation, Operations, Sustainment, and NetOps activities Support DISA CIO in extending the Enterprise Information Environment (EIE) technical procedures and standards for DISA internal systems PEO – GES Data Services Office Additional questions, please use the Comments and Feedback link at http://metadata.dod.mil

  8. “I need a tank” What does it mean? Make Data Understandable One reason Government Agencies and Military Services have trouble operating jointly is that they speak different languages. Registering and using taxonomies improves precision search and recall Register Terms, Definitions, & Relationships

  9. DoD Metadata MgmtEngineering-level Processes Namespace Managers & WGs Namespace Managers & WGs DISA Data Services Office Namespace Managers & WGs Governs/Coordinates Within Namespace Operates, Maintains DoD Metadata Registry Hosts Participates in Consults, Subscribes to & Submits to/Downloads from Metadata Registry Discusses Participates in DoD Metadata WG DOD Developer • DoD MDR Focus Group • DDMS Focus Group • Taxonomy Focus Group

  10. Distributed Configuration Management 7 “Super” Namespaces MDR Governance Namespaces

  11. DoD MDR v7.0 Public areas Public areas

  12. Login

  13. Advanced Search

  14. Search Results ratings Add to Briefcase

  15. Information Resource Details User-defined URLs

  16. Navigate through Relationships

  17. MDR Support for Mediation tacrep.xml ebXML Registry Query service schemaLocation=“.../tacrep.xsd” tacrep.xsd (schema) HasXSLT tacrep-2-kml.xsl (stylesheet) MDR Organizes Components for Mediation notional tacrep.kml Leverage Registered Components & Associations 17

  18. DoD Metadata Registry (MDR)https://metadata.dod.mil Purpose Provide an on-line repository which enables developers to reuse, understand, integrate with, and share existing data assets (metadata) Targeting web services, databases, and vocabularies Mandated for the publishing of semantic and structural metadata DOD Directive 8320.2 Provides a portal for developer access and web services for machine-to-machine access Key Facts Over 8,000 users and 180,000 assets registered Over 900 Programs of Record supported Serving the DoD, DHS, IC, NASA and NATO Hosted on NIPR, SIPR, and JWICS (in progress) User driven via DoD Metadata Working Group and Feedback links Implements the ebXML standard for Metadata Registries (in DISR) User Name/Password or Single Sign-On through DKO Governance structure provided by Joint Enterprise Services ERB Primary Benefits Enables reuse and governance of data assets Foundation for other services; e.g. mediation Allows the COI data assets to exist after the COI disbands Primary Audience DoD Capability Developers, COIs

  19. DoD MDR History DoD XML Registration Memo MDR Implements ebXML RIM & RS DoD Metadata Registry MDR on SIPR COE XML Registry v6.1 v7.0 v5.0 v2.1 v4.1 v3.1 v1.0 2000 2002 2004 2006 2008 IC Info Sharing Data Standards Coordination Activity (DSCA )

  20. MDR Users • Developers • Re-use and subscribe to registered data components, and/or register new ones they have created • Community of Interest (COI) Metadata Governors • Configuration Manage (CM) registered component (e.g., posting new metadata versions, version change notification etc.) • Acquisition Policy Makers • Use Metadata Registry metrics for acquisition oversight (e.g., reflecting program participation, specific data component re-use etc.) • Applications • Interface with registered components via Metadata Registry Web Service (ebXML) to exploit reference tables, transformations, and XML schemas at Run Time

  21. MDR Size and Scope • Instances on 2 security enclaves • Unclassified and Secret • Users – 10,186 • Information Resources • Submission Package – 1,263 • Schemas (e.g., DTD, XSD, etc) – 4,814 • Translations (i.e., XSL, XSLT) – 180 • Services (i.e., WSDL) – 555 • XML samples – 251 • Taxonomies (i.e., OWL) – 178 • Reference Data Sets – 5,709 • Amplifying Documents (e.g., DOC, ER1, etc) – 5,268 • 900 Programs of Record, 700 Organizations Unclassified MDR Inventory as of 8 Oct 2008

  22. Making Data Visible, Accessible, and Understandable through Metadata Federated Search Interface NCES Enterprise Catalog Defense Online warfighter 3 Register discovery metadata IAW DDMS in Federated Search-enabled Catalog Audience: warfighter, intelligence, business user Audience: developer, application program NCES Service Registry 2 Register service metadata & endpoints Uses registered resources developer Web Interface (HCI) 1 DoD Metadata Registry Construct & Submit Package to MDR. Contains WSDL, schemas, stylesheets, taxonomies & any other pertinent information Web Service Data Store MDR Submission Pkg

  23. Net-Centric Publisher Service Endpoint XSD WSDL DDMS Simplifies Metadata Publishing

  24. Practical Utility for Taxonomies • Applicable for • Content • Services • Structural Metadata • Aid precision search via DoD Discovery Metadata Specification (DDMS) Country.sql ISO.xsd FIPS.xsd <ddms> : <Subject>…/DoDCore#Terrorist_Event</Subject> </ddms>

  25. MDR Support for Taxonomies DoD Core taxonomy Organization PoliticalOrganization UrCOI taxonomy MyCOI taxonomy Group TerroristOrganization equals ForeignTerroristOrganization TerroristGroup equals al-Qaeda AlQaida Consumer View Producer View <ddms> : <Subject>…/MyCOI.owl#AlQaida</Subject> </ddms> 25

  26. DoD Core Taxonomy v0.75c

  27. UCore v2.0 Default Taxonomy

  28. FEA-BRM as OWL-encoded Taxonomy

  29. Framework for Registry Interoperability   Federated Queries Inter-Registry Object References Enterprise Search Aggregator Service MDR UDDI MDR DDMS-based Query Service Architecture Products  Centralized Authority w/Local Caching Object Relocation More attention needed MDR (U) MDR Unclassified fully automated push  Online Package Registration (U) MDR (S) Secret Top Secret (U) MDR (TS) (S) USTRANSCOM USMTF Adapted from OASIS Cooperating Registries

  30. Useful Links • DoD CIO Data Strategy Homepage • http://www.defenselink.mil/cio-nii/datastrat/ • DoD Metadata Registry • http://metadata.dod.mil • DoD Discovery Metadata Specification • http://metadata.dod.mil/mdr/irs/DDMS • NCES Techguide • http://metadata.dod.mil/mdr/ns/ces/techguide/ • NCES Developer Community • https://www.us.army.mil/suite/page/384284 • COI Toolkit • https://www.us.army.mil/suite/page/479547 • Intellipedia • https://www.intelink.gov/wiki

  31. UNCLASSIFIED www.disa.mil UNCLASSIFIED

  32. “Documents” Documented via Keyword, Text, RSS/Atom, DDMS Metadata Cache (traditional) Search Index, OpenSearch (RSS/Atom) Syndication Feed, Google Data API (DDMS) Metadata Catalog, Federated Search Service Web Services Documented via WSDL + XSD(s) Metadata Cache ebXML Registry UDDI Registry Styles SOAP REST Make Data Visible REST REST = HTTP/HTTPS + URI + XML

  33. DoD Discovery Metadata Specification (DDMS) IC: ISM Configuration managed by DoD Metadata WG ISO: Dublin Core * ISO: Dublin Core * ISO: Dublin Core * ISO: Dublin Core * ISO: Dublin Core W3C: Date & Time Data Catalog (historical) ISO: Dublin Core ISO: Dublin Core ISO: Dublin Core ISO: Dublin Core W3C: OWL DDMS endorsed by Executive Order 13388 * OGC: GML “Further Strengthening The Sharing Of Terrorism Information To Protect Americans” W3C: Date & Time ISO: Dublin Core * mandatory ISO: Dublin Core ISO: Dublin Core DDMS: Leverages Industry Standards

  34. Utility Beyond “Make Data Visible” DoD Discovery Metadata Specification (DDMS) Configuration managed by DoD Metadata WG Make Data Accessible (responsibly) * IC: ISM * ISO: Dublin Core * ISO: Dublin Core Enable Data to be Trusted Enable Data to be Trusted ISO: Dublin Core * ISO: Dublin Core ISO: Dublin Core Data Catalog (historical) W3C: Date & Time ISO: Dublin Core Make Data Understandable Make Data Understandable ISO: Dublin Core ISO: Dublin Core ISO: Dublin Core W3C: OWL DDMS endorsed by Executive Order 13388 * OGC: GML “Further Strengthening The Sharing Of Terrorism Information To Protect Americans” Make Data Accessible (practically) Make Data Accessible (practically) W3C: Date & Time * mandatory ISO: Dublin Core ISO: Dublin Core ISO: Dublin Core DDMS is enabler to multiple Data Strategy Goals 34

  35. DDMS Home Pagehttp://metadata.dod.mil/mdr/irs/DDMS Current Version: 2.0

  36. Federated Search Use Case EC FS FS FS FS FS FS FS FS FS COI, POR, C/S/A Data Sources populated from applications, databases, web content, etc. Capabilities: Interfaces: Enterprise Services – DECC Columbus Community of Interest Program of Record EC Enterprise Catalog DS DS DS DS External Applications, Services, and Data Sources FS Federated Search DS DS DS For immediate discoverability users may post metadata to the Enterprise Catalog Query is federated and results returned. Users DS Data Source NCES Fed Search Aggregator Enterprise Web Content is crawled and indexed NCES Security Service NCES Service Discovery NCES Enterprise Catalog User Authorized by NCES Security Services Fed Search Aggregator discovers data sources from Service Discovery Aggregated results returned User Submits Search Query DECC Columbus & San Antonio DS (Web Enabled) DS (Web Enabled) User Logs into DOL Portal, DKO, or COI Application Results viewed by user NCES Enterprise Services Management Federated Search enables information sharing within and between PORs and COIs

  37. Considerations Practical & Responsible Application & Browser Documents & Systems Make Data Accessible ICISM = IC Information Security Metadata RBAC = Role-based Access Control ABAC = Attribute-based Access Control

  38. Attribute-Based Access Control Releasability Rules classification="TS" ownerProducer="USA GBR" SCIcontrols="SI TK" FGIsourceProtected="ISR" disseminationControls="OC REL PR" releasableTo="USA AUS CAN GBR" declassDate="2002-08-01" derivedFrom="Source Document" Notional Example Security Clearance: S Citizenship: US Rank: Major AOR: AF

  39. Enterprise Metadata Initiatives • Senior Enterprise Services Governance Group (SESGG) [DoD & IC] • DoD Metadata WG • DoD Metadata Registry (DoD MDR) • DoD Discovery Metadata Spec (DDMS) • Service Registry & Governance WG (SR&G WG) • Content Discovery IPT • Ucore v2.0 [DoD, IC, DHS, DoJ] • Automated Metadata Population Service (AMPS) [AF lead] • XML Cross Domain Services [DISA-NSA] • Controlled Unclassified Information (CUI) – [Federal]

  40. UCore V2.0 Vision, Scope and Governance Executive Steering Council (ESC) (Rotating Chair between DoD, IC, DoJ, DHS and State/Local Representative) DoD CIO Initial Chair Development and Configuration Management (DoD, DoJ Lead) Policy Implementation (IC Lead) Outreach and Communications (DHS Lead) Business Case Design / Build Pilot Test and Eval Config Mgmt http://www.ucore.gov

  41. Automated Metadata Population Service (AMPS) Pilot Outcomes Annotators for DDMS elements Creator Description Date Format Geospatial Identifier Security Subject Title Type Annotators for Asset Types Microsoft Office PDF Position-Location Indicators Information Exchange Schemas HTML ID: http://www.jfcom.mil/cac_v1.doc Title: Cyber Awareness Campaign Creator: Jim Smith Subject: cyber warfare; information assurance; non-kinetic effect; security; cyber threat; infrastructure Description: A campaign to increase awareness of cyber threats to the DoD information infrastructure. Security: U//FOUO Type: Cyber COI Date: 2008-03-31 Format: Microsoft Word Geospatial: 35.8, -75.3 COI Vocabularies AMPS Information Asset

  42. Differentiating Types of Metadata Structural & Semantic Metadata “Rules governing a chunk” - Name, description, data constraints, and relationships of tags used in information resources to delimit one chunk of data from another chunk Artifacts where structural metadata is described: XML schemas, RDBMS structures, taxonomies Register in DoD Metadata Registry, use submission pkg Resource Metadata “Advertisement” - Terms to aid in the recall and retrieval of artifacts Artifacts that we collect resource metadata on: PPT, DOC, GIF, JPG, MPG, RDBMS Register in “Data Catalog”, use DoD Discovery Metadata Specification (DDMS)

More Related