NextGen Network-Enabled Weather (NNEW) Registry/Repository

1. NextGen Network-Enabled Weather (NNEW)Registry/Repository Oliver Newell, Kajal Claypool 5 August 2008

2. Overview • Registry/repository background • NNEW registry/repository use case overview • Registration of 4-D cube data domain taxonomy • Registration of a dataset • Basic dataset and service provider discovery • Enhanced discovery using Ontologies

3. Key Roles of a NNEW Registy/Repositry • Build-time • Storage/discovery of service interface descriptions (WSDL and associated schemas) • Storage of dictionary/vocabulary information (e.g. coordinate reference system dictionary, upper-level weather ontology) • Run-time • Discovery of datasets and their associated data access services using high-level metadata • Dataset metadata within registry includes weather cube domain ‘membership’ (e.g. SAS) • Dataset can be a member of more than one domain

4. NNEW Registry/Repository • Universal Description, Discovery, and Integration (UDDI) Registry • Underpowered with respect to NNEW use cases • Limited query capabilities (especially geospatial queries) • Industry support on the wane… • IBM, Microsoft (co-creators of UDDI spec) closed public registries in 2006 • IBM now offers more capable (but proprietary) registry/repository solution • OASIS UDDI Technical Committee now inactive • ebXML Registry/Repository • More flexible, extensible solution • Freely-available version (freebXML) used in variety of domains • DOD Metadata Repository uses freebXML ‘under the hood’ • NNEW Approach • Adopt ebXML registry/repository – extend as needed • Partner with ebXML registry/repository vendor (Wellfleet) • Formalize extensions with OASIS ebXML technical committee

5. Key features of ebXML RegRep Registration, Discovery, Queries ebXMLRegRep Taxonomies, Classifications, Relationships Federated Queries, Inter-registry links SOA RegistryRepository Federated Information Manage-ment Standard Metadata Cataloging, Validation, Version Control,Lifecycle Support,Extensible Info Model Secure Architecture SOA Governance Digital Signatures,Audit Trail,Access Control,SAML SSO Events Content-Based Event Notification

6. DOD Metadata Repository Web GUI • Based upon freebXML Registry OS project • Customized web based GUI client • Manages schemas, data elements, attributes, document type definitions, style-sheets, data structures • Not currently used for registration of services for run-time discovery purposes • ebXML reg/rep does support the run-time use case Source: <https://metadata.dod.mil/mdr/about.htm>

7. NNEW Registry-Repository/Catalog • For 2007/2008 demos, using early release of ebXML RegRep 4.0 – compliant regrep (OASIS draft standard expected ~4th Qtr 2009)

8. Data Discovery and Access using OGC Services and ebXML Registry (Dec ‘07) MIT Lincoln Laboratory CIWS Forecast Data Catalog (ebxml) Web Coverage Service (OGC WCS) FAA Technical Center National Center for Atmospheric Research (NCAR) “Discover Available Data Sets” “Get data from time T1 to T2 in spatial region A and return in NetCDF format using a Lambert Conformal Projection” Icing Turbulence Data Web Coverage Service (OGC WCS) NOAA Surface Observations Data Requested Data OGC service-enabled display Client Web Coverage Service (OGC WCS) NOAA NCAR MIT/LL Virtual 4-D Weather Cube

9. Overview • Registry/repository background • NNEW registry/repository use case overview • Registration of 4-D cube data domain taxonomy • Registration of a dataset • Basic dataset and service provider discovery • Enhanced discovery using Ontologies

10. Creation of 4-D Cube Single Authoritative Source Domain “The designated administrator for the SAS domain accesses a 4-D Wx Data Cube registry and determines the identifiers of all datasets categorized as the SAS for their respective data type. A list of all the datasets is passed to the Enterprise service manager responsible for the top-level information registry. The service manager verifies that all the selected datasets have the appropriate access rights and services for the SAS domain, and registers the domain in the registry for subsequent use by all consumers of SAS information.” Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3

11. Creation of 4-D Cube Experimental SAS Domain “NCAR, NASA, NOAA, and MIT Lincoln Laboratory are collaborating on a research project to produce a new weather data product. The 4-D Wx Data Cube data access mechanisms are used by the project due to its distributed nature. The majority of inputs that are used to create the new product are those provided by the SAS, with the exception of a higher-quality surface winds product. A new ‘Experimental SAS’ is generated for the duration of the research project, and made available in a registry. Following the completion of the project, the ‘Experimental SAS’ designation is removed from the product’s metadata registry.” Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3

12. Build-Time Dataset/Service Discovery “A small flight services company serving the state of Alaska wishes to incorporate a summary of winds information into an existing ‘quick-look’ Web page that they provide to general aviation users. The company browses the NextGen registry to discover what types of wind data are available, and what services exist to access them. An appropriate data access service is located, and members of the company’s software team download the service and data format schema needed to construct a data access client. A thin layer of additional logic is used to convert the raw data to the desired ‘synopsis’ form, and the updated Web application is made available to pilots over the open Internet for flight planning purposes. The data access client software, which conforms to a service standard used to disseminate many types of weather data, provides the company with easy access to numerous other types of weather data in the future.” Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3

13. Run-Time Dataset/Service Discovery “A client application contacts a registry containing dataset information for the weather cube, and requests information about ‘air temperature’ datasets available in the SAS domain. Metadata for the single, primary, ‘air temperature’ data set are returned, along with metadata for zero or more backup datasets, each with a priority indication describing the ordering in which the backups should be used. For each dataset, metadata describing one or more data access services, along with the data formats supported by each service, are returned. The client application selects the most appropriate service for the primary dataset, and begins accessing data.” Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3

14. Overview • Registry/repository background • NNEW registry/repository use case overview • Registration of 4-D cube data domain taxonomy • Registration of a dataset • Basic dataset and service provider discovery • Enhanced discovery using Ontologies

15. 4D Weather Data Cube Domains 4D Weather Cube Domains • Single Authoritative Source (SAS) • Regulatory2a – Government2b – Commercial • Both 1 & 2a • Intermediate • 4a – Government • 4b – Commercial • SAS data (supports ATM decisions) to be available on open & unrestricted conditions • Opportunities for commercial entities to provide weather information still exist • Flexible support for different domains needs to be incorporated into architecture

16. Importing the Cube Domain Classification Scheme

17. Weather Cube Domain TaxonomyebRIM Encoding (some details ommitted) <RegistryObjectList xmlns="urn:oasis:names:tc:ebxml-regrep:xsd:rim:4.0" xmlns:rim="urn:oasis:names:tc:ebxml-regrep:xsd:rim:4.0"> <rim:RegistryObject xsi:type="rim:ClassificationSchemeType" id="urn:ogc:def:ebRIM-ClassificationScheme:FAA:DataCubeDomains" isInternal="true" nodeType="urn:oasis:names:tc:ebxml-regrep:NodeType:UniqueCode"> <rim:Name> <rim:LocalizedString xml:lang="en" value="FAA/NOAA/DOD Data Cube Domain Taxonomy" /> </rim:Name> <rim:ClassificationNode code="DataCube" parent="urn:ogc:def:ebRIM-ClassificationScheme:FAA:DataCubeDomains" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube"> <rim:Name> <rim:LocalizedString xml:lang="en" value="DataCube" /> </rim:Name> <rim:ClassificationNode code="Restricted" parent="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:Restricted"> <rim:Name> <rim:LocalizedString xml:lang="en" value="Restricted" /> </rim:Name> <rim:ClassificationNode code="Government"> ... </rim:ClassificationNode> <rim:ClassificationNode code="Commercial"> ... </rim:ClassificationNode> </rim:ClassificationNode> <rim:ClassificationNode code="Unrestricted" parent="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:Unrestricted"> <rim:Name> <rim:LocalizedString xml:lang="en" value="Unrestricted" /> </rim:Name> <rim:ClassificationNode code="SAS"> ... </rim:ClassificationNode> <rim:ClassificationNode code="Regulatory"> ... </rim:ClassificationNode> </rim:ClassificationNode> </rim:ClassificationNode> </rim:RegistryObject> </RegistryObjectList

18. Viewing the Cube Domain Taxonomy via the RegRep UI

19. Overview • Registry/repository background • NNEW registry/repository use case overview • Registration of 4-D cube data domain taxonomy • Registration of a dataset • Basic dataset and service provider discovery • Enhanced discovery using Ontologies

20. Datasets and Metadata Metadata Metadata Dataset Metadata Dataset Series e.g. Precip Data for most recent one year period Registry/Repository Dataset Series Metadata Operates On Service Metadata Data Access Service OperatesOn dataset association(s) can be added at RegistrationTime via registry UI if not included in Service Metadata Data Access Service (e.g. JMBL, WCS, WFS)

21. ISO 19115/19139 Metadata

22. ISO 19119 Service Metadata Class Diagram (from specification) • ISO 19119 (Conceptual Model) does not yet have a standardized schema (ala the ISO 19115/19139 pair)

23. OGC Cat-ebRIM Dataset/Service Registry Information Model Mapping • Using ebRIM mapping for Service Metadata temporarily until standard 19119 schema is available

24. Registry/Repository Basic Profile Supports ISO 19119 Service Taxonomy • JMBL, WFS, WCS services can all be classified under this scheme • Flexibility to declare datasets available via JMBL and/or OGC WFS/WCS

25. INSPIRE Geoportal Dataset/Service Discovery (ISO 19115/ 19139)

26. INSPIRE Geoportal Metadata EditorISO 19115/19139 • INSPIRE portal does not (apparently) support a service interface • ebXML regrep UI being designed to support the functionality of the INSPIRE portal, plus the ebXML registry service interface (ebRS)

27. Overview • Registry/repository background • NNEW registry/repository use case overview • Registration of 4-D cube data domain taxonomy • Registration of a dataset • Basic dataset and service provider discovery • Enhanced discovery using Ontologies

28. Dataset Discovery: All Datasets in Cube

29. Dataset Discovery: All Restricted Datasets in Cube • Lightning is only dataset series classified under DataCube/Restricted – all other datasets not returned by query

30. Dataset Discovery: All Unrestricted Datasets in Cube • All datasets with the exception of lightning returned by query • Follow-up queries allow determination of services that operate on each returned data set (may be zero or more)

31. Overview • Registry/repository background • NNEW registry/repository use case overview • Registration of 4-D cube data domain taxonomy • Registration of a dataset • Basic dataset and service provider discovery • Enhanced discovery using Ontologies

32. Intelligent Discovery of Datasets • Intelligent discovery • Consults knowledge base/ontology to find alternative meanings • Clustered by: synonyms, parent, children • Enables discovery of resources without exact keyword match

33. Intelligent Discovery of Datasets WCS Dataset: air_temperature ebXML Registry/Repository Ontology Editor WCS air_temperature Service Provider JMBL Dataset: temperatureAir Domain Expert Ontology Mapper JMBL temperatureAir Service Provider WCS Dataset: surface_air_temperature Query Editor SPARQL Query Engine WCS Architect surface_air_temperature Service Provider Service Discovery UI/Client Service Consumer • Supports discovery of datasets and associated datasets using CF standard names, JMBL parameter names, or potentially other terms • Flexibility to support JMBL and/or OGC service access to the same dataset

34. Registry/Repository and Infrastructure Support ebXML Registry/Repository Ontology Editor Domain Ontology Service Description Service Annotation Service Annotator UI Domain Expert Ontology Alignment Mapping Ontology OWL-S Service Description Generator Query Editor SPARQL Query Engine Architect Service Producer Service Discovery UI/Client Service Consumer

35. Discussion • Is an ebXML registry/repository potentially useful in the Joint METOC context? • Would it be complementary to JMCAT ? • Would it largely replace JMCAT functionality? • What is DOD’s view of ISO 19115/19139 metadata vs. DDMS?