2013 NOAA MADIS Plans MADIS – The Meteorological Assimilation Data Ingest System

1. - - - - - - - + o X P R Surface Aircraft Radiosonde Profiler GOES Satellite POES Satellite Radiometer 2013 NOAA MADIS Plans MADIS – The Meteorological Assimilation Data Ingest System Greg Pratt, Leon Benjamin, Thomas Kent, Gopa Padmanabhan, and Leigh Cheatwood NOAA/Oceanic and Atmospheric Research/Earth System Research Laboratory/Global Systems Division Tim Mcclung, Steven Pritchett, Curtis Marshall, Ben Kyger, Daniel Starosta, Rebecca Cosgrove, and Wen Meng NOAA/National Weather Service/Office of Science and Technology & National Centers for Environmental Prediction John Bates, Drew Saunders, and Philip Jones NOAA/National Environmental Satellite, Data and Information Service/National Climate Data Center NOAA Mission The MADIS Team MADIS Final Operating Capability NOAA’s mission increasingly demands advanced data management processes, including data integration, to achieve interoperable, accessible, and readily usable observational data. NCEP Internet NWS/OST&NCEP Transition/Operations Tim Mcclung, Steven Pritchett, Curtis Marshall, Ben Kyger, Daniel Starosta, Rebecca Cosgrove, and Wen Meng WOC Decoders Providers Users Ingest MADIS Goal NESDIS/NCDC Archive John Bates, Drew Saunders, and Philip Jones A more usable, complete, accurate, timely, and higher density observational infrastructure for use in local weather warnings and products, numerical weather prediction, and use by the greater meteorological community. Integration and Quality Control Distribution Archive OAR/ESRL/GSD Transition/R&D/Conduit to Operations Greg Pratt, Leon Benjamin, Thomas Kent, Gopa Padmanabhan, and Leigh Cheatwood MADIS Provides NCDC • Access to real-time and archived data sets • Uniform data formats, observation units, and time stamps • Observational Quality Control (QC) • Network-enabled distribution with server-site sub-setting • Authentication for proprietary data • User documentation and help desk support ESRL/GSD MADIS software is being ported to a distributed environment hosted at NOAA’s WOC, NCEP, and NCDC with GSD supporting a research-to-operations test bed hosted at GSD. Providers Observations and Metadata http://madis.noaa.gov/mesonet_providers.html MADIS Users Include… • International meteorological centers • >200 universities • Public • NWS Forecast Offices and National Centers • OAR, NESDIS, NOS • NCAR, NASA, DOE, FAA, DOT • Hundreds of private companies MADIS Data Sets MADIS Future Plans • Reach Full Operating Capability (FOC) for real-time processing within the NWS • Work with NESDIS to transition the MADIS archive capabilities into operations at NCDC • Work with Federal Highway Administration Clarus team to fully transition Clarus capabilities into MADIS • FAA NextGen/NWS AWIPS data discovery/dissemination capable • Conduit for efficient transfer of research and development to operations National Mesonet (NM) Mobile Platform Environmental Data (MoPED) Clarus Available Observations MADIS Google Displays MADIS observations covering North America • Additional data sets – (FAA 1 minute ASOS , Clarus, …) • Advanced data query and web services • Improved data/metadata standards • Improved QC and station monitoring • Open source development environment • Extend QC algorithms to meet operational and research needs MADIS Data Scope • 66,127 stations from over 160 surface networks producing nearly 13 million observations per day • 154 Profiler sites (>200,000 observations per day) • >450,000 aircraft observations per day • Plus global radiosonde and satellite observations Surface Temperatures Metadata SensorML

2. - - - - - - - + o X P R Surface Aircraft Radiosonde Profiler GOES Satellite POES Satellite Radiometer A 2011 Update to the NOAA MADIS System MADIS – The Meteorological Assimilation Data Ingest System Greg Pratt, Leon A. Benjamin, Randall S. Collander, Thomas B. Kent, and Gopa Padmanabhan NOAA/Oceanic and Atmospheric Research/Earth System Research Laboratory/Global Systems Division Patricia A. Miller and Michael F. Barth NOAA/Oceanic and Atmospheric Research/Earth System Research Laboratory/Global Systems Division (Retired) Steven Pritchett, Curtis Marshall, David Helms, Rebecca Cosgrove, Daniel Starosta, Michelle Mainelli, Mahnaz Dean, and Robert Bunge NOAA/National Weather Service NOAA Mission MADIS Overall Benefits MADIS Real-Time Processing is Now Operational Within the NWS NOAA’s mission increasingly demands advanced data management processes, including data integration, to achieve interoperable, accessible, and readily usable observational data. A more usable, complete, accurate, timely, and higher density observational infrastructure for use in local weather warnings and products, model predictions, and hazardous situations. MADIS NWS Compute Environment NCEP TOC MADIS Goal MADIS Central U.S. Observations Decoders Ingest To integrate other-agency observations with NOAA obser-vations and make them easily accessible and useable to the meteorological community. Internet Integration and Quality Control Distribution MADIS Provides • Access to real-time and saved data sets • Uniform data formats, observation units, and time stamps • Observational Quality Control (QC) • Network-enabled distribution with server-site subsetting • Authentication for proprietary data • User documentation and help desk support ESRL/GSD MADIS software has been ported to an integrated NWS TOC and NCEP distributed environment with a supporting backup and research-to-operations test environment at GSD. Standard Surface Observations Additional MADIS Surface Observations MADIS – New Datasets All MADIS Observations • New MADIS datasets are now in development at GSD: • National Mesonet observations with extended metadata; • Mobile Vehicle observations. MADIS Users Include… MADIS Data Sets • International meteorological centers • >200 universities • NWS Forecast Offices and National Centers • OAR, NESDIS, NOS • NCAR, NASA, DOE, FAA • Hundreds of private companies MADIS Provides QC and Station Monitoring Support Observations from a mobile platform reporting via the Mobile Platform Environmental Data (MoPED) System on December 24, 2010. The plot shows low temperature errors at high platform speed, but higher errors at lower speeds. Available Observations MADIS Future Plans MADIS observations covering North America • Reach Full Operating Capability (FOC) for real-time processing within the NWS • Work with NESDIS to transition the MADIS archive capabilities into operations at NCDC • Continue research and development enhancements MADIS Data Scope • 69,917 stations from over 160 surface networks producing nearly 13 million observations per day • 154 Profiler sites (>200,000 observations per day) • >450,000 aircraft observations per day • Plus global radiosonde and satellite observations Data Provider Notified Data Restored • Additional data sets • Advanced data query and web services • Expanded metadata fields • Improved QC and station monitoring Estimated observation errors for a station in Galveston, Texas. The time series shows a bias detected by MADIS and corrected by the data provider.