The NOAA Operational Numerical Guidance System: Meeting the WoF and WRN Challenge

1. William. M. LapentaActing DirectorEnvironmental Modeling CenterNOAA/NWS/NCEPGeoff DiMego , John Derber , Yuejian Zhu , Hendrik Tolman , Vijay Tallapragada, Shrinivas Moorthi , Mike Ek , Mark Iredell , Suru Saha The NOAA Operational Numerical Guidance System: Meeting the WoFand WRN Challenge

2. NOAA Center for Weather and Climate Prediction: “A Game Changer” A.K.A.—the new building…. • Four-story, 268,762 square foot building in Riverdale, MD will house 800+ Federal employees, and contractors • 5 NCEP Centers (NCO, EMC, HPC, OPC, CPC) • NESDIS Center for Satellite Applications and Research (STAR) • NESDIS Satellite Analysis Branch (SAB) • OAR Air Resources Laboratory • Includes 465 seat auditorium & conference center, library, deli, fitness center and health unit • Includes 40 spaces for visiting scientists • Represents a “Game Changer” in our ability to do business

3. NOAA Operational Numerical Guidance Supports the Agency Mission Lead Time and Accuracy! • Numerical Weather Prediction at NOAA • Related to ability to meet service-based metrics (below) • National Weather Service GPRA* Metrics (* Government Performance & Results Act) • Hurricane Track and Intensity • Winter Storm Warning • Precipitation Threat • Flood Warning • Marine Wind Speed and Wave Height • Customer Service Provider • Operational numerical guidance provides foundational tools used by Government, public and private industry to Improve public safety, quality of life and make business decisions that drive US economic growth 3

4. DRAFT Storm Prediction Center • Potential Products and Services

5. DRAFTStorm Prediction Center • Desired Numerical Guidance Attributes • Inside every HRRRE member • Movable domain update SSEF: • 1 km movable regional domain • 20-30 member storm scale ensemble • advanced state-of-the-science DA • run every 1 hr with forecasts to 18-24 hr • focused on “severe weather of the day” areas 5

6. Convergence of NAM, RAP, HRRR and SREF between 2016 and 2018 System must meet requirements of today and future Provide NextGen Enroute AND terminal guidance Provide PROBABILITY guidance with full Probability Density Function specified Address Warn-on-Forecast as resolutions evolve towards ~1 km Core elements Common data assimilation system Code Infrastructure and management system Potential Attributes of the North American Rapid Refresh Ensemble (NARRE) Hourly updated with forecasts to 24 hours Control data assimilation cycles with 3 hour pre-forecast period 84 hr forecasts are extensions of the 00z, 06z, 12z, & 18z runs Plans for a National Mesoscale Ensemble System

7. NOAA’s OperationalNumerical Guidance Suite Oceans HYCOM WaveWatch III Forecast • NOS – OFS • Great Lakes • Northern Gulf of Mex • Columbia R. • Bays • Chesapeake • Tampa • Delaware Climate Forecast System Hurricane GFDL HWRF Coupled GFSMOM4 NOAH Sea Ice Sea Nettle Forecast ~2B Obs/Day Satellites + Radar 99.9% Dispersion ARL/HYSPLIT Regional NAM WRF NMMB Regional DA Global Forecast System Global Data Assimilation Severe Weather WRF NMM/ARW Workstation WRF Short-Range Ensemble Forecast Space Weather North American Ensemble Forecast System Regional DA WRF: ARW, NMM NMMB Air Quality GFS, Canadian Global Model ENLIL NAM/CMAQ Rapid Refresh for Aviation 7 7 7 NOAH Land Surface Model

8. Numerical Guidance Suite Execution on the Operational NOAA Supercomputer 24-h Snapshot 20 August 2012 Number of Nodes High Water Mark 2010 SREF 12 18 GFS 00 06 00 NAM CFS GEFS Time of Day (UTC) 8

9. Operational Compute Resource Consumption (%) by Component • Statistics accumulated over a 24-h period • Total 64.9% of production machine • NAM, CFS and SREF compose 29.7% of production Percent of Total CPU Numerical Guidance System Component 9

10. Global Data AssimilationSystem Upgrade Implemented 22 May 2012 • Hybrid system • Most of the impact comes from this change • Uses ensemble forecasts to help define background error • NPP (ATMS) assimilated • Quick use of data 7 months after launch • Use of GPSRO Bending Angle rather than refractivity • Allows use of more data (especially higher in atmos.) • Small positive impacts • Satellite radiance monitoring code • Allows quicker awareness of problems (run every cycle) • Monitoring software can automatically detect many problems • Partnership between research and operations • (NASA/GMAO, NOAA/ESRL, Univ OK, and NOAA/NCEP) • Consolidation across systems • Unify operational data assimilation system for global, regional and hurricane applications • Cost effective—O&M • Configuration management

11. NCEP Closing the International GapJune, July, August 500hPa Geopotential RMSE 2011 2012 Meteo-Fr CMC NCEP UKMO ECMWF Solid line lower than dashed indicates improvement between 2011 and 2012 NCEP Only System to show improvement between 2011 and 2012 • NCEP achieved significant improvement in 2012 for day 3 and beyond • NCEP is now similar to UKMO skill in this metric and AC

12. Gridpoint Statistical Interpolation(GSI) Data Assimilation System • Unified variational data assimilation (DA) system • Global and regional applications • Weather and climate • Operational system being used by • NOAA (GFS, NAM, RTMA, HWRF, RAP…) • NASA (GMAO global) • AFWA (Testing in progress) • Distributed development: • NCEP/EMC, NASA/GMAO, NOAA/ESRL, NCAR/MMM, … • A community code used in NOAA operations and research • Supported by NCEP, NASA, DTC • Disciplined code management policy and procedures required to incorporate changes across user organizations http://www.dtcenter.org/com-GSI/users/index.php

13. Goals of Community GSI Efforts • Provide current operational GSI capabilities to the research community (O2R) • Provide a framework for distributed development of new capabilities & advances in data assimilation • Provide a pathway for data assimilation research to operations process (R2O) • Provide rational basis to operational centers and research community for enhancement of data assimilation systems

14. Adding New AnalysisVariables to GSI • No longer a difficult procedure • Thanks to GSI Bundle infrastructure (via NCEP/NASA collaboration) • Generalized structure for analysis, guess, state, and control variables • e.g., hydrometeors, vertical velocity, double moment microphysics variables, etc. • Control through table in the anavinfo file • Greatly reduces the number of routines needed to be modified • Modify table(s) + possible additional model I/O steps + observation ingest + forward operator + background error • Can also disable the conventional large-scale balance constraints present in the NMC derived background errors

15. North America Model (NAM) Implemented 18 October 2011 NEMS based NMM Outer grid at 12 km to 84hr Multiple Nests Run to ~60hr 4 km CONUS nest 6 km Alaska nest 3 km HI & PR nests 1.3km DHS/FireWeather/IMET (to 36hr) Rapid Refresh (RAP) Implemented 1 May 2012 WRF-based ARW Use of GSI analysis Expanded 13 km Domain to include Alaska Experimental 3 km HRRR NOAA Operational Mesoscale Modeling for CONUS: WRF-Rapid Refresh domain – 2010 RUC-13 CONUS domain Original CONUS domain Experimental 3 km HRRR

16. Numerical Guidance to Support a NWS Operational Warn on Forecast Capability S. Lord – NWS/OST and NCEP/EMC Staff V4.0

17. Introduction • Goal: field an operational, credible and skillful WoF capability by 2025 • Combines forecast requirements for • NWS severe weather forecasts and warnings • Aviation (NEXTGEN) • Daily general weather guidance to 3 days, including QPF • Closely allied with Hurricane forecast technology (HFIP) • Effort exceeds decade-long development and implementation cycle • Current operational system provides look at the future pieces • Incremental system development provides a risk-managed path for operations • With sustained development of the operational pieces, and sufficient HPC, NOAA (OAR and NWS) can achieve the WoF goal • Development of operational systems provides enhancement and generalized capabilities to incorporate R&D and improve capabilities (including forecast skill) • Improved skill must be documented with appropriate performance measures (TBD) • HPC capability growth is essential to meet goal • Adequate resolution for predicting high impact weather events poses largest operational HPC requirements • Forecast model • Data assimilation • Ensemble & postprocessing system • Advanced nature of severe weather problem requires larger ratio of R&D HPC to operational HPC • Constant $$ HPC growth and new technology (e.g. GPU, MIC, etc) will be necessary but insufficient

18. International Thrust – UKMO 2014 • UK is a very small country! • Area covers approximately Fire Wx nest (next slide) • Weather events less diverse • Regional high resolution NWP is more affordable computationally • US global system (planned for 2014) has comparable resolution • SL T1148 (17 km) • Hybrid DA (27 km ensemble) enhances skill to comparability with UKMO • US regional system lags in operational implementation (due to HPC constraints) but not technology level

19. Global  Continental  Local Nesting Strategy • Discussion items • Nesting necessary to save computing; current NAM domain could be expanded to global 10 km • Advantages/disadvantages of same model for global and regional • Applicability (DA) of a multi-model ensemble-based covariance estimate with multiple bias characteristics • Data assimilation at resolution of 3 km or less poses challenges • Advanced assimilation of radar reflectivity • Consistent integration of radar and satellite info • Proper dynamically balanced analysis increments at multiple high resolution scales (10, 3, 1 km) to minimize spin-up/spin-down and maximize evolution of nascent disturbances

20. Numerical Guidance Attributes (1) • Analysis and uncertainty products • Hourly refresh but sub-hourly for rapidly changing observables (reflectivity, hydrometeors, wind shear) • 1 km resolution for rapidly changing observables • Hybrid analysis provides analysis uncertainty information and ensemble perturbations • Multi-model ensemble-based background (technology not developed) • Analysis variables • Surface (sensible wx) variables • Clouds • Radar reflectivity, hydrometeors • 3-D atmosphere • Aerosols • Skin temperature • Precipitation amount and hydrometeors • Land & hydrology • Eventual coupled system to add marine & coastal analysis (waves, water level, water quality) • Products support forecaster Situational Awareness (SA) • MRMS products are an initial capability • Enhancement to Rapidly Updating Analysis (RUA) • Combines obs. from • Radar • Satellite • In situ • Expands to hydrological and coastal marine domain • Provides best and consistent geophysical picture from all available information • Catchup cycle for late data produces “Analysis of Record” for verification

21. Numerical Guidance Attributes (2) • Initialized from hourly analysis • Evolved through RAP, HRRR & NAM development • North American Rapid Refresh Ensemble (NARRE) • High Resolution Rapid Refresh Ensemble (HRRRE) • 1-way nested • Technology available in NAM as prototype • 1 km resolution where needed • Ensemble-based probabilistic products support • Severe convective weather (incl. tornado environment) • Aviation and surface transportation • Landfalling hurricanes • Flash floods • Eventual support for • Storm surge • Waves • Water quality • High resolution analysis and nested forecast system provide consistency between real-time SA and forecast evolution

22. Warn on Forecast (WoF) System Comparison Summary

23. System Evolution HPC Gaps

24. Runtime & optimal node apportionment for operational NMMB nesting with a Fire Wx nest over CONUS (72 nodes) 12 km parent 8/72 or 11% 6 km Alaska nest 5.72 or 7% 1-way nests solved simultaneously WRF-NMM takes 3.6 times longer than NEMS-NMMB to run Fire Wxnest Increased nest capability by adding processors! 4 km CONUS nest 40/72 or 56% 1.33 km CONUS FireWx nest 12/72 or 17% 3 km Hawaii nest 4/72 or 5% 3 km Puerto Rico nest 3/72 or 4% 24

25. Summary • Ensemble-based probabilistic guidance for WoF is a major challenge • Scientific development • Demonstration of system capabilities • Current operational capabilities can provide basis for future system • But many changes necessary in response to R&D needs and results • HPC is second major challenge • Prototype system exceeds planned availability by factor of 4-6 • R&D may suggest compromises and where to spend HPC resources to achieve required skill affordably

26. Supplementaries

27. North American Rapid Refresh ENSEMBLE (NARRE) NMMB (from NCEP) & ARW (from ESRL) dynamic cores Common use of NEMS infrastructure and GSI analysis Common NAM parent domain at 10-12 km Initially ~6 member ensemble made up of equal numbers of NMMB- & ARW-based configurations Hourly updated with forecasts to 24 hours NMMB & ARW control data assimilation cycles with 3-6 hr pre-forecast period (catch-up) with hourly updating NAM & SREF 84 hr forecasts are extensions of the 00z, 06z, 12z, & 18z runs – for continuity sake. SREF will be at same 10-12 km resolution as NARRE by then SREF will have 21 members plus 6 from NARRE for total of 27 NARRE requires an increase in current HPCC funding DiMego 2015-2016

28. High Resolution Rapid Refresh ENSEMBLE (HRRRE) Each member of NARRE contains 3 km nests CONUS, Alaska, Hawaii & Puerto Rico/Hispaniola nests The two control runs initialized with radar data & other hi-res obs This capability puts NWS/NCEP[+OAR/ESRL] in a position to Provide NextGenEnroute AND Terminal guidance (FWIS-like) Provide PROBABILITY guidance with full Probability Density Function specified, hence uncertainty information too Provide a vehicle to improve assimilation capabilities using hybrid (EnKF+4DVar) technique with current & future radar & satellite Address Warn-on-Forecast as resolutions evolve towards ~1 km NAM nests are extensions of the 00z, 06z, 12z & 18Z runs. HRRRE requires an increase in HPCC funding over and above that required for the NARRE DiMego 2017-2018