Areas for National and International Collaboration Toward Net-Centric Weather Information Access

1. Areas for National and International Collaboration Toward Net-Centric Weather Information Access Joint Action Group for XML and Web Services Omaha, NE July 2007

2. Areas Addressed for Collaboration • Developing a work plan to resolve issues • Version control • Conceptual Data Model (UML vs. ER) • Data dictionary • XML schemas • Access services • W3C/OGC standards • General access by major data types • Specific access for major customers • Standards followed • Must be non-proprietary

3. WMO/ICAO Issues(Not to be Distributed) • Overall concept is to reduce risk • Developing the standard is to reduce risks to customers • Financial – different sets of equipment for US/Europe • Without US position the WMO dictates the standard • But there is still the risk that we develop to the US standard but then have to readapt to a standard that the WMO puts forth • Provide leadership to the international community • Partnership with EUROCONTROL provides leverage with WMO for adoption of standard

4. EUROCONTROL Meeting Issues • Significant issues for data exchange • Version control • Conceptual Data Model (UML vs. ER) • Data dictionary • XML schema • Access services • W3C/OGC standards • General access by major data types • Specific access for major customers • Non-proprietary standards • Many of these have been addressed to considerable extent by existing US programs

5. Version Control • Includes software versioning and operational support to backward/forward compatibility • This is driven by the need for agility over time, especially as standards evolve • Existing services use XML Schema 1.0 • XML Schema 1.1 will provide inherent version support • Not yet released • Is expected to lower version control costs • What entity will host and fund version control support?

6. Conceptual Data Model – Data Storage • A uniform conceptual data model of database design across DOD and non-DOD has not yet been decided upon and may not be necessary or desirable • DOD has such a conceptual data model • Joint METOC Conceptual Data Model (JMCDM ) • Uses ER representation • Gridded data, observations, climatology, imagery, platforms, space weather, alphanumeric bulletins, remote sensed observations, catalog, … • Representation of the model • DOD uses ER representation • DOD is investigating translation from ER to UML • Report is expected to be available late August 2007 • Some non-DOD agencies and EUROCONTROL use UML

7. Conceptual Data Model – Data Exchange • A conceptual model of Web Services interface exchange messages is desirable • Schema exist from which these models could be generated • DOD - JMBL • NOAA - DWML and GML • …

8. Conceptual Data Model - JMBL Experience • Approach started with “top down” approach • Initial work was development of an ER database model • The data model was then used as a starting point to develop the JMBL data exchange schema • This interface schema abstracted the database design • Approach evolved to “bottom up” approach • Examined user need to revise JMBL • Use cases can assist with this approach

9. Data Dictionary • There are several technologies available to capture a data dictionary • Dictionaries, taxonomies, ontologies, … • DOD • Master Parameter List (MPL) identifies all parameters available at the JMBL interface level • Working on a taxonomy • NOAA • Could develop the equivalent of the Master Parameter List from DWML • Unidata • Uses netCDF CF convention as a data dictionary • Lacking in sensor, satellite and radar • A standard should be considered across the community

10. XML Exchange Schema • DOD – Joint METOC Broker Language (JMBL) • Schema for data request and response • Gridded data, observations, climatology, imagery, platforms, space weather, alphanumeric bulletins, remote sensed observations, … • Initial operational capability at AFWA, FNMOC and NAVO • Gridded data, observations, alphanumeric messages, platform information • Phased implementation in progress for additional METOC data types • NOAA – Digital Weather Markup Language (DWML) • Schema for data request and response • Forecasts and observations • In use at NOAA • NOAA – Common Alerting Protocol (CAP) • Schema for data response • Watches, warnings and advisories • This is an alerting protocol in use by NWS but not specifically a weather protocol • FAA/NWS – Aviation Digital Data Service (ADDS) • Schema for data response • NWS text observations, forecasts and advisories specific to aviation

11. Access Services • Relevant standards • W3C,OGC, … (elaborated on following slide) • Exchange schema addresses • General access by major data types • Specific access for major customers • Finest granularity possible • Compliance beyond basic W3C standards depends on requirements derived from use-cases • Follow through with work plan will be necessary to evaluate relevant standards • Standards followed must be non-proprietary

12. Relevant Standards • W3C Standards • XML, XML Schema, SOAP, WSDL, RDF, OWL (Semantic Web) • OASIS • ebXML registry/repository • Web service profile • OWL profile • UDDI registry • Common Alerting Protocol (CAP) • ISO TC/211 Geographic information standards • ISO 19101: Geographic information - Reference model • ISO 19103: Geographic information - Conceptual schema language • ISO 19107: Geographic information - Spatial schema • ISO 19108: Geographic information - Temporal schema • ISO 19109: Geographic information - Rules for application schema • ISO 19110: Geographic information - Methodology for feature cataloging • ISO 19111: Geographic information – Spatial referencing by coordinates • ISO 19115: Geographic information - Metadata • ISO 19118: Geographic information - Encoding • ISO 19123: Geographic information - Schema for coverage geometry and functions • ISO 19136: Geographic information - Geography markup language (GML) (Shared OGC standard) • ISO 19139: Geographic information - Metadata - XML schema implementation • ISO 8601 Time

13. Relevant Standards (cont’d) • Open Geospatial Consortium • Geography Markup Language (GML), a toolkit for building data descriptions • SensorML • Catalog Service (ebXML profile) • Web Feature Service (WFS) • Web Coverage Service (WCS) • Web Map Service (WMS) • Web Processing Service (WPS) • DOD • NCES catalog service • JMBL • DDMS metadata standard • Based on Dublin Core • Incorporates IC/ISM security markup • FAA/Eurocontrol • Aeronautical Information Exchange Model (AIXM) • Weather Exchange Model (WXXM) • Models and notional data dissemination mechanisms based on ISO/OGC standards

14. Existing Access Services & Compliance • DOD – JMBL • W3C, OASIS compliant • XML 1.0, SOAP, WSDL, Schema, WS-I • NOAA –DWML • W3C compliant • XML 1.0, SOAP, WSDL, Schema • NOAA – CAP • W3C, OASIS compliant • XML 1.0, CAP1.0 • FAA/NWS – ADDS • W3C compliant • XML 1.0, REST

15. Sample Capabilities • JMBL • Gridded atmospheric (areas, points, lines, slant angle) and ocean forecasts, observations (raw, decoded, selected parameter), alphanumeric messages (METARS, TAFS, PIREPS, AIRMETS, SIGMETS, …), station info • DWML • Forecasts (point, selected parameters/character of the day), city forecasts & observations, national highs/lows • CAP • Text warnings, watches, advisories, outlooks, alerts • ADDS • METARS, TAFS, PIREPS, AIRMETS, SIGMETS, station info • WCS in development for icing, turbulence, winds, temperatures, ceiling, visibility, echo top, and VIL

16. What type of services? • Data exchange/use cases • Grids/obs/warnings • JAG will collect and define use cases to identify current and near-term capabilities • Sample template follows • Level of granularity: • One system level slide • Two or three user task level slides • Data retrieval by end-user • Optionally include an OV-1 graphic for system level use case • Target delivery date is October 1, 2007 • Email use cases to Ken Barnett and cc to Tim Hopkins • Focus on aviation use cases • Can we recommend alignment of current systems to achieve some goal… how do we leverage?

17. Simple (Trivial) Text-Only Use Case Example(From W3C Schema versioning use cases) Web services content" scenario C: original instance with new schema • Brief summary: An instance of the original schema will be accepted by the new schema. • Basic course of events: • 1. An instance of the original schema is processed. • 2. The processor identifies it as a version of the new schema. • 3. The system processes the instance. • Desired outcome: • The processor determines that it is a version of the new schema. • Valid against the new schema. • PSVI returns all information items expected from the new schema (even though it is an old instance). • This scenario may occur when an old client invokes a new release of the service.

18. General Form for Use Cases • Goal: • Paragraph (or two) describing background and goal of use case at high level • Level: • Level of the use case. The exact terminology used is somewhat flexible. A use case summarizing a behaviour at a high level uses ‘Summary’, while a more detailed user-level behaviour specifies ‘user’. See reference for list of possible and their meanings. • Preconditions: • Set of conditions that are assumed to be true at the start of the use case. This allows the detailed steps that follow to focus on only those steps that are important to the particular use case. • Main Success Scenario: • 1. First step in scenario. Steps are usually one or two sentences in length • Second step…. • Third step…. Generally the number of steps is between 3 (for fine-grained use cases) and 20, but this is not fixed. • Etc…. • Extensions: • 3a. Alternate path for step 3. Alternate paths can describe handling of a particular error condition, or any key variation of the use case that is deemed relevant with respect to the requirements it imposes on a design

19. Example: Access of Satellite Imagery • Goal: • Access satellite imagery data for a given region of interest at a specific time. Returned imagery is in coordinate reference system chosen by user • Level: • User • Preconditions: • Satellite data of interest exists in database on the ground and is accessible via a LAN with sufficient bandwidth to download the image. The user has the necessary security clearance to access the data. • Main Success Scenario: • User brings up geospatial data analysis tool. • Tool dynamically discovers available data sources (catalogs) • User makes request for metadata describing all satellite imagery from yesterday within 500 miles of Borneo • Tool presents user with list of available data sets, their accompanying metadata, and the capabilities of the data access services available to access the data (available formats, coordinate reference systems, etc…) • User makes request for data of highest interest, specifying the desired spatial, temporal, and coordinate system properties. • System accesses the raw data image, produces the desired image, and returns it (or a reference to it) to the user • Image is displayed in geospatial data analysis tool. • Extensions: • 4a. No imagery currently exists for target area. A request for imagery is submitted to the entity responsible for scheduling satellite imagery data collections. • 4b. The status (accepted/pending, rejected) of the request is returned to the user

20. Preliminary System List JAG – XML & Web Services

21. Follow On • Next meeting • Virtual meeting with Web Ex or Go To Meeting • Target is October 15, 2007 • Ken to set up a test meeting ahead to identify participation issues • Agenda • Discuss use cases to identify baseline of current and near-term capability • Outline a business case for resources to establish • Standardized, XML-based schema exchange format • Consistent, national net-centric weather services interfaces • Pick a date for follow on face to face meeting

22. Roadmap Objective • Shape JMBL into JMBL++ to become a national and international standard for machine-to-machine environmental data exchange • Evaluate other technologies for adoption as special purpose services within JMBL++ • Leading candidates include DWML, CAP, ADDS, etc. • E.g., propose CAP to for alert messaging for the meteorological community • The overall approach is driven by use cases

23. Roadmap • Brief OGC meeting at NCAR (Sep 07) • Use Case Gathering (Oct 07) • Derive use cases from selected existing systems • Augment with near-term use cases • ROM cost to this point is roughly 440 hours (senior analyst/engineer) • Obtain resources to work remainder of roadmap (Oct 07) • ROM cost will depend on outcome of use case analysis • Additional team representation will be needed • Use Case Abstraction (Nov 07) • Identify commonality • Consolidate

24. Roadmap • Service Design (Feb 08) • Assess how well existing services (JMBL, DWML, CAP, ADDS, …) • Meet needs of use cases • Comply with standards such as • ISO (19xxx), OGC (GML), HDF5, NetCDF • Some of these are data model 'building blocks' and require application profiles • Apply use cases to shape JMBL into JMBL++ to become a national and international standard for environmental data exchange • Seek close EUROCONTROL collaboration on conceptual data model, … • Identify functionality gaps in existing services, missing services • Augment existing services to fill functionality gaps • Work within standards bodies working groups • Create service 'profiles' for weather COI • Propose new services as needed • 'Flight path service', etc. • Welcome use cases from EUROCONTROL for further collaboration

25. Roadmap (cont’d) • Beyond Feb 08 • Address registry, discovery, metadata, catalog, … • File format issues (netCDF-4/binary xml) • Examine REST as a complement to SOAP • Common business rules for server behavior

26. WXXM (like) Application Schema GML CAP XML Schema JMBL++ Data Model ISO 19xxx Standards XML-based Formats Data Model/ Building Block Existing Format/ Building Block COI-developed Profiles Building Application-Level 'Profiles'

27. Data Format Layers NetCDF-4-CF-WX? NetCDF-4-CF HDF5-EOS-WX? HDF5-EOS NetCDF-4 HDF-5 HDF-5 Binary Formats … Data Model/ Building Block Existing Format/ Building Block COI-developed Profiles

28. Backup

29. POCs

30. Areas for national collaboration.(* = Areas we expect will come up at the Aug 07 meeting with EUROCONTROL.) • Use Service Oriented Architecture (SOA) approach (tri-agency agreement) • *Setting an action plan to resolve issues • Catalog • Discovery – taxonomy • Subscription • CAP- warning advisory service • Shared security model • *Version control • *Conceptual Data Model (UML, ER) • Exchange data formats • NetCDF4 (CF), HDF-5 (EOS) • *Data dictionary

31. Areas for national collaboration.(* = Areas we expect will come up at the Aug 07 meeting with EUROCONTROL.) • *XML schemas • *Access services • *OGC standards • *W3C standards • *General access by major data types • *Specific access for major customers • Formats supported in delivery • Common business rules • Communication challenged users (bandwidth and protocol) • *Standards followed • Must be non-proprietary • Preserve independence from physical data base

32. Deliverable approach • Develop pieces to build business plan • What are the resources needed? • Staff • Level of technical effort • Time lines for projects/demos • Areas to focus on – JMBL, OGC standards, etc • * WMO/ICAO angle • * Technology args • Net-centric • What standards: W3C/OASIS/ISO/OGC • What level interop/validation? • Standardize business rule profile? • Must be non-proprietary • * Short term – EUROCONTROL support

33. SWIM JMB NOAA NNEW NCAR MITRE MIT/LL NOAA/ GSD NCDC & CTRS Boeing DOC AFWA CNMOC NESDIS NCEP(9) NWS • OGC Services • GML-based data format • netCDF4, HDF5 grids • OWL ontologies • architectural documents • Use cases • AWIPS • NDFD • Web services • NWSTG • GML-based data format • netCDF4, HDF5 grids • OWL ontologies • architectural documents • Use cases • Websphere (& related technologies)