NOAA NextGen Wx Information Database (WIDB) IT Architecture Overview System of Systems Workshop

1. NOAANextGenWx Information Database (WIDB) IT Architecture OverviewSystem of Systems Workshop NWS Office of Science and Technology Systems Engineering Center (Skjei Telecom) Mike Asmussen

2. Outline • Introduction • Requirements Development • Requirements approach • Key documents • Requirements summary • Generalized IT architecture • Assumptions • Architecture efforts to date • Wx Products • FAA IT Architecture • NWS / NOAA IT Architecture Summary • Next Steps

3. Introduction

4. Key Concepts • NWS / NOAA architecture will follow a System of Systems approach • No one entity is building THE CUBE • It will be a federation of network-enabled services (some existing services, and some yet to be deployed) • Each service will be “registered” in a federated registry/repository, which • Will provide descriptive information about each available service (via published metadata) and, • Will point any interested service user to the appropriate service endpoint to allow service access

5. Requirements Development

6. Requirements Approach • Requirements Challenges • Most available requirements are high level (not IT requirements) • No single document serves as the “definitive” requirements source • Some requirements not clearly characterized as: • IOC/MOC/FOC • Single Authoritative Source (SAS) / non-SAS • Approach • Surveyed numerous JPDO and NOAA documents • Determined relevant IT related requirements (as well as derived IT requirements) • Categorized requirements • Created table of requirements • Where possible, assigned timeframe of deployment and whether requirement is SAS-specific • Identified Weak Areas • Only limited IT performance and security requirements have been developed to date • IOC Cube contents are still under consideration (required use cases still being examined)

7. Key Documents

8. Requirements Summary • Access • Aggregate overlapping requests • High BW and low BW access methods support • Agility • Ability to add new systems, services, products, data fields, users, etc. w/o system interruptions • Support legacy and new systems together • Scalable over time • Archival / logging • Past transaction retrieval (e.g., for accident investigations) • Availability • Fault tolerance • Load balancing • Essential FAA service support - .999 • Critical FAA service support - .99999 • SAS (essential): • MTTR - 0.5 hr, • MTBF – 5,000 hrs • Outage max – 10 mins • SAS (critical): • MTTR - 0.5 hr • MTBF – 50,000 hrs • Outage max – 6 secs

9. Requirements Summary (cont) • Compatibility • With CWSUs, AWC, WFOs, Tower systems, TRACON systems, ARTCC, ATCSCC • With FAA architecture • Contents • Support Wx Products required for aviation purposes, for example: • NOAA provided, FAA-provided, 3rd party provided • North American and global • Forecasts model data (probabilistic) • Sensor products • Radar, lightning, satellite, aircraft sensors, airport, ground, ocean, air, METARs • Observations – PIREPS • Advisories, watches, warnings • Climatological data • Formats • Grid based (machine readable) where possible (e.g., NETCDF4) • Encoded versions of legacy text products (via WXXM or JMBL) • Otherwise, text and graphic? Binary? • Data Consistency • Deconflicted SAS (temporal and spatial) • Data Formats / Protocols • Allowing ease of exchange • Standards based (e.g. HTTP, XML, SOAP, OGC) • Discoverability (metadata/registry/ repository) • Products/data • Formats • Access methods • Associated services

10. Requirements Summary (cont) • Distributed (not centralized) • Ease of integration with aviation decision support tools • Governance • Access control • Standards • Common ontologies • Business rules • SAS decisions • Intelligent processing • Data interpolation • Netcentric • SOA based • System of systems • Legacy system support (along with new systems) • QOS / Performance • Varies by user / use case / function • Request management • Request / reply exchange mechanisms • Data for time period of choice • Data for geographical construct of choice • Product / data field of choice • Priority • Desired format / translations • Compression

11. Requirements Summary (cont) • Security • Data security • Physical / network security • Field by field, product by product access control • Subscription management support • SAS management and governance • System management • Failure detection / switchover • Load balancing • Health monitoring / analysis / trending / logging • Users • Aviation (primarily), governmental, commercial, military, international • Non-aviation, research, NOAA-internal • Verification / quality control

12. IT Architecture Introduction

13. Assumptions • The following key assumptions are driving NOAA’s NextGen IT Architecture: • Use of a System of Systems approach • Compatibility with key NOAA and NWS Enterprise Architecture guidance • Compliance with NextGen requirements (with flexibility to evolve as requirements evolve) • Supports IT ConOps Use Cases • Compatible with evolving FAA Architecture • Supportive of NextGen Enterprise Architecture definition of Business Services / Operational Activities

14. Architecture Efforts to Date • Compiling requirements / use cases • Identifying required Wx Cube data (and candidate source / destination systems), flows, and formats • Reconciling IOC Product list , FAA Product Flows spreadsheet, SA Plan, others • Working closely with FAA Architecture team to ensure architecture compatibility • FAA architecture approach – Hub and spoke / Store and forward • Federated registry / repository (for metadata) • FAA Origin Servers (ingest/store/serve FAA data) • FAA Distribution Servers (cache/handle requests of FAA users) • FAA External Distribution Servers (handle requests to and from systems externally to FAA) • Based on WCS/WFS Reference Implementation (RI) being developed by MITLL, NCAR, GSD which will be available for NOAA re-use • Reconcile JMBL vs. WXXM • Working with GSD to evaluate use of JMBL – several JMBL RIs may be available for NOAA use • Standards decisions • Non-gridded • JMBL vs. WXXM (or both) for NOAA (FAA is standardizing on WXXM) • Gridded data formats • NetCDF4/HDF5, GRIB2? Others? • Web Services / message exchange standards (e.g, WCS, WFS, JMBL, SOAP, XML, WMS? etc) • Metadata standards • Beginning to work with NOAA/NWS weather system owners to ensure compatibility of IT architecture / migration approach • NWS IT System Architecture document and IT System Design document to be developed and released by March 2010 (with interim drafts prior)

15. Sample Wx Products List for NextGen Inclusion • MISs • LAMP output • Tropical cyclone bulletins • FAs • Tornado watch / warnings • Severe thunderstorm watch / warning • Convective outlook • Non-convective watches, warnings, advisories • Freezing level analysis • Surface analysis • High level SIG Wx • Verification statistics • Mid level SIG Wx • Low level SIG Wx • Winds aloft • RUC outputs • WRF-RR outputs • HRRR outputs • NCWF • NCWD • GTG • FIP • CIP • Global wind grids • ASOS OMOs • Others? • NEXRAD Level III • Lightning data • MADIS (TBD subsets) • GOES data • METSAT data • TAFs • METARs/SPECIs • SIGMETS / G-SIGMETS • AIRMETS / G-AIRMETS • Surface fronts • Meso-scale boundaries • 3-D reflectivity products • CCFPs • CWAs

16. Resultant Potential Candidate Systems for Cube Inclusion • NDFD • NDGD • NESDIS • GAS (GOES R) • NDE (NPOESS) • Non-NOAA systems • NEVS (RTVS, Stats on Demand) • NEXRAD (Radar Data Server) • NSSL Mosaics • RIDGE radar servers • TOC (NWSTG/NOAANET) • Web Farms • Other Sources for? • Canadian Radar • Puerto Rico model data • STMAS data • UKMET model data • ADDS • AWIPS • CAWS • GIS system • IOOS • IRIS • Lightning Data Services • MADIS • MDCRS • NCDC • NCEP/NOMADS • GFS (Global Forecast System) • HRRR • LAMP • NAM (North American Mesoscale) • RUC (Rapid Update Cycle) • WRF-RR

17. FAA Architecture Summary

18. Layered Architecture Approach

19. End Users Internal FAA Architecture Concept Consumer Cube Service Adaptor (CCSA) Consumer Systems Net-Enabled Services ProviderSystem Registry/ Repository Origin Servers (with System Ingest Adaptor and Provider Data) Distribution Servers IP Network 4-D Wx Data Cube

20. Internal FAA Architecture

21. NWS/NOAA IT Architecture Summary

24. Definitions • Cube Input Edge Server (CIES) • Allows remote access to the weather data (or subsets thereof) via WCS/WFS/other web services. • Provides for the ingest of weather data required by the Cube (obtained either directly from the native source or via a Service Adaptor – which is mostly likely case for most data sources initially) • Performs the necessary processing and local storage • Cube Output Edge Server (COES) • Provides for the request and retrieval of Cube data from remote WCS/WFS/other web services • Performs the necessary processing and local storage • Allows access to the data by the requesting local destination system. • Service Adaptor (SA) • Source Service Adaptor (SSA) - Performs processing to: • Transform native or legacy source wx data that is required for publishing to the Wx Cube into a format appropriate for ease of access via Cube Input Edge Servers (e.g., transformed into one of several supported standards) • To make source wx data available to Cube Input Edge Servers via a convenient and reliable network accessible means (e.g., Web Services-based communication) where such a means may not currently exist. • Destination Service Adaptor (DSA) - Performs processing to: • Transformwx data from a format appropriate for ease of access by Cube Output Edge Servers into a native or legacy format compatible with the destination system • A Service Adaptor may be optional, where all such SA processing may instead be performed directly by a Cube Input Edge Server or Cube Output Edge Server or internal to the source or destination system themselves (for future systems). • Optional Accessible Storage • Optional storage to temporally hold wx data before being ingested into the cube or before being processed for use by the requesting destination system

26. Example Architecture Use Cases • Assumptions • Reg/rep has already been queried in all cases • Does not reflect any security aspects • Use Case examples: • Requestor-FAA, Data Source – NOAA, Data Type – NonGridded (WFS) • Requestor-FAA, Data Source – NOAA, Data Type – Gridded (WCS) • Requestor-NOAA, Data Source – FAA, Data Type – NonGridded (WFS) • Requestor-NOAA, Data Source – FAA, Data Type – Gridded (WCS) • Requestor-NOAA, Data Source – NOAA, Data Type – JMBL • Requestor-FAA, Data Source – NOAA, Data Type – NonGridded (JMBL) • Requestor-FAA, Data Source – NOAA, Data Type – Gridded (JMBL)

34. Example Architecture Use Cases • Assumptions • Reg/rep has already been updated in all cases • Does not reflect any security aspects • Use Case examples: • Loading data into Cube: Source System to CIES via SSA • Loading data into Cube: Source System to CIES using Optional Storage • Loading data into Cube: Source System to CIES Directly

39. Example Architecture Use Cases • Assumptions • Reg/rep has already been queried in all cases • Does not reflect any security aspects • Use Case examples: • NOAA Cube request via DSA (Intermediate Stand-alone or Bolt-on SA) • NOAA Cube request with optional storage (Intermediate Stand-alone or Bolt-on SA) • NOAA Cube request from Destination System (SOA via External COES) • NOAA Cube request directly to Data Source (Embedded COES)

41. Centralized Options

42. Redundancy Options

45. Architecture Next Steps • Identifying definitive source for each Cube WX product • Defining Cube “format” for each Wx Product • Resolving JMBL/WXXM standards decision (or decision to support both) and finalizing all exchange formats / protocols / standards • Involving stewards of each Cube candidate system in determining details of “connecting” to the Cube • Refining requirements / use cases (performance, security, verification, varied QOS offerings, external interfaces/system compatibilities, weather products needed) • Determining key telecommunications infrastructure requirements, interfaces, and implementation details • Finalizing high level NOAA Cube IT architecture (while ensuring compatibility with FAA architectural approach) • Developing design for Cube edge servers • And a million other things to do.