The Evolution of AWIPS

1. The Evolution of AWIPS Jason Tuell Office of Science and Technology National Weather Service

2. Overview • What is AWIPS? • Case For Change • Where AWIPS is going? • Roadmap for AWIP evolution • Summary

3. What is AWIPS? • AWIPS is the integrating element of the National Weather Service Modernization • Integrates data from modernized observations systems, NEXRAD, ASOS, GOES into single processing and visualization environment • Hardware, communications, software

4. AWIPS System Overview(Functional) • Primary Field Systems (135 sites) • *122 Weather Forecast Offices (WFO) • issue all routine forecasts and severe weather watches and warnings • **13 River Forecast Centers (RFC) • Provide hydrologic and flood prediction and monitoring at the river basin level • National Centers and HQs (20 systems) • *7 National Centers (NC) • **6 Regional Headquarters Offices (RHQ) • **2 HQ Operational Systems (NWSHQ) • 4 Training Center Systems (NTC) • 1 National Lab System (FSL) • Development and Test Systems (14 systems) • 6 HQ Test and Development Systems (NWSHQ) • 3 Special Office Systems • *1 Network Control Facility System • also have off site backup facility, BNCF • 4 Prime Contractor Test Systems • 169 Total Systems * 24x7x365operations ** Occasional 24x7 operations during severe weather events.

5. AWIPS System Elements • WFO Configuration • 3-6 workstation class computers per site. • Standard servers, storage and peripherals at all sites • Standard WFO software suite (Custom+COTS) • RFC Configuration • 6-10 workstation class computers per site • More RAM and disk storage on standard servers/storage devices. • Specialized “River Ensemble Processor” compute cluster at each site • Specialized RFC software suite (Custom+COTS)

6. Major Features of AWIPS System Architecture • Highly Distributed • Nearly identical hardware/software suite at all sites/systems • Highly Redundant • Most data is distributed via Satellite Broadcast Network and stored redundantly at all sites/systems • Operational WFO’s provide site backup services to designated neighbors, in case of site failures. • Most hardware w/in each site is redundant as well. • Highly Available • 24x7x365 operations at all WFO’s, combined with mission requirements for rapid distribution of watches and warnings, affecting life and property, drive system availability requirements. Critical system components require 99.9% availability. • High Performance • During severe weather operations, minutes and seconds matter, thus end-to-end system performance is paramount, especially during peak loads.

7. WFO/RFC Hardware Architecture (~July ’05) WAN Linux CP LINUX LX LINUX XT Network Attached Storage (NAS) Linux AX Router Nexrad RPG Ethernet 1000/100/10 Mbps SWITCH Serial MUX Plaintree Switch LINUX PX1 LINUX PX2 LINUX DX1 LINUX DX2 RFC REP LDAD Firewall 100Mbps FDDI Legend LDAD SERVER HP-UX 10.2 < 1 yr old 2-3 yr old DS1 HP-UX10.2 DS2 HP-UX10.2 1-2 yr old 3+ yr old

8. AWIPS Communications • Satellite Broadcast Network (SBN) • Provides broadcast and reliable multicast data transmission to field sites. • Transmitted data includes: Centrally collected radar data, GOES imagery, NCEP model data, field observations, and watches and warnings • DVB-S • Single channel solution. • Linearly scalable up to 43 Mbps, on demand • AWIPS Terrestrial WAN • Dual homed, redundant, inter-meshed, hierarchal hub and spoke frame relay network • Carries radar product data from all WFOs for central collection by the NCF for dissemination over the SBN • Carries re-transmission frame/product requests to the Network Control Facility (NCF) from WFOs for nonreceived SBN data • Carries forecast collaboration traffic between adjacent WFOs • Carries any other traffic deemed “operational.

9. Future changesAWIPS Communications • AWIPS terrestrial WAN will be consolidated, along with all other NOAA line office WANs, into a single network solution • Consolidating bursty data into one network increases overall network efficiency. • The new single network will be an MPLS IP VPN. • Next generation replacement to obsolete frame relay. • VPN any-to-any architecture replaces inefficient hub-spoke architecture.

10. AWIPS Software • > 4.5 million SLOC • Mixed Languages – • C/C++, FORTRAN, Python, Java, Perl, Tcl/TK, X and Motif • Mixed operating systems • HP UX 10.20 and Red Hat 7.1+/7.2 • Moving to all Linux environment built on Red Hat Enterprise 3.0 • Relational data base • Postgres/ Informix • Migrating off Informix • Legacy architecture • Main architecture design dates from early to mid 90s • 95% Government developed software • 5% prime contractor software

11. AWIPS Software • D2D – visualization environment • Warning Tools • Warngen • Watch-Warning Advisory (WWA) • Riverpro • Forecast preparation tools • GFE - Grid editing and formatters • AVNFPS – TAF preparation • Decision Assistance Tools • SCAN – severe weather • FFMP – flash floods • SAFESEAS - maritime • Infrastructure • Ingest, decode and storage • Informix • Communications routines

12. Nexrad Radar Acq server EXT DISSEM NWWS NWSRFS WHFS HIPS or other sat format convert IFPS NCEP WAN Sat Comms Router GRIB SBN (TG) TG Acq server LDAD TextDB external providers IPC (sockets) Text METAR, RAOB, + SBN GOES W GOES Acq server Data Storage & Mgmt (netCDF, flat files. Informix) external recipients Radar SBN GOES E Text Workstation Localization LAPS D2D (Meteorological Analysis and Display) Site Monitoring MHS apps IPC (socket) IPC (socket) AWIPS MC (remote monitoring) AWIPS MHS (message handling system) DNS extension AWIPS Software Architecture

13. The Case for Change • Checkpoint Analysis done Fall 2004 on AWIPS hardware, communications, software and data • AWIPS solid operational system, but ill poised to meet future operational demands • Architecture challenged to meet increasing data volumes, collaborative requirements and needs to accelerate the transition of research to operations

14. Hardware Compute Platforms Data Storage Devices LAN/WAN/SBN Interfaces Peripherals Architecture Documentation Data Inputs Product Improvement Plans Requirements Outputs Archives Architecture Product Improvement Plans Communications Infrastructure WAN SBN Architecture Product Improvement Plans Software Operating System Off The Shelf (Commercial and/or Open Source) Baseline AWIPS Performance Management and Control Requirements Local Applications Performance Management and Control Architecture Documentation Product Improvement Plans Cost effectiveness Present State: Summary Checkpoint Analysis

15. Where is AWIPS going? • New contract awarded August 2005 - Raytheon • Performance based, firm fixed price contract • Contract transition August - October 2005 • Five base years with five options years • Contract components • Core O&M • Network Control Facility Operations • Satellite Broadcast Network Operations • Software Integration and Test • Sustaining Engineering • Software maintenance option • Continuous Technology Refresh (CTR) option • Hardware & communications improvement • Software re-architecture

16. Software Re-architecture • AWIPS moving to Service Oriented Architecture (SOA) • Raytheon to implement J2EE Enterprise Service Bus • Raytheon to deliver development environment and software development kit to support software migration and development • SOA will provide a more flexible and robust infrastructure for AWIPS

17. Approach to Migrating AWIPS to a New Operational Concept FY-08 PAC AWIPS Evolution Budget Initiatives AWIPS for the 1990s AWIPS for the year 2010 AWIPS Re-Compete O&M Cost Savings funding migration to an SOA • Supports New Ops Concept • More flexible in Production/Delivery • Increased AWIPS/NAWIPS Integration • Improved RTO efficiency • Increased access to data for decision making • Reduced software O&M costs • Flexible data delivery • WFO Centric Architecture • Little AWIPS/NAWIPS Integration • High software Maintenance Costs • Poor RTO efficiency • Fire Hose Data Distribution A Services Oriented Architecture is necessary but not sufficient to get us to a new Operational Concept and a more flexible AWIPS

18. AWIPS’ Needs • Data delivery and information architecture • Introduce a more flexible data retrieval paradigm • Visualization • Provide a consolidated foundation for visualization of data and products within the AWIPS environment • Collaboration • Provide an infrastructure for collaborative services with internal NWS, NOAA and external trusted partners • Information Generation • Provide an infrastructure for the generation of NWS products and services

19. Data Delivery/Information Architecture • Move to “push-pull” data delivery paradigm • Expanding AWIPS beyond push capability (SBN) only • Exploring use of OpenDap as a technology to enable a push-pull paradigm • Business and performance cases to dictate final implementation • Delivering all the data still may be most cost effective solution • Latency may make “pull” only approach impractical to support the warning mission

20. Visualization • Common visualization tools needed • “Incompatible” visualization environments being used between applications • Lack of common look and feel • Possible solution • “Plug in” architecture, based on Forecast Systems Laboratory advanced prototyping • Potential benefits of reduced migration costs with increased flexibility

22. Collaboration • Goals • Provide infrastructure for real time graphical collaboration between WFOs, RFCs and centers for enhanced internal collaboration • Provide infrastructure for real time collaboration with other NOAA entities • Provide infrastructure for collaboration with trusted partners, e.g., Emergency Managers • Tools • Leverage current tools such as FX-Net (low bandwidth, “AWIPS on a laptop” and FX-C) • Chat, Whiteboarding, remote briefing capabilities

23. Information Generation • Needs • Standardize infrastructure for generation of NWS products and services • Enable more rapid adoption and integration of new dissemination technologies • Outcomes • Reduction of number of unique product templates • More responsiveness to customer driven changes in products

24. Information Generation Content Generation Product Generation Metadata Forecasts (GFE) Grids Web Warnings(GHG, WARNGEN, RiverPro) GIS? NWR XML? NWWS Other Other

25. AWIPS Evolution Outcomes • Increased integration of AWIPS and NAWIPS • Improved research to applications throughput • Increased access to all environmental data for decision making • Reduction of software O&M costs and reduced tech refresh costs • Increased flexibility to seamlessly transfer operational functions and responsibilities between WFOs and National Centers for new concepts of operation

26. Challenges and Risks • Migration of operational system to new architecture • Changing the wheel on the car at 60 mph • Performance requirements • Defining and measuring performance requirements against which to measure new architecture • Training of management, developers to work in new environment

27. Summary • AWIPS moving to a Services Oriented Architecture • Under CTR option, Raytheon will deliver new architecture and infrastructure under the new contract • Migration of current baseline and future baselines to be done FY10 • Enhancement of AWIPS capabilities within new architecture critical to meeting current and future mission needs