Assessing Uncertainties in Observations , Analyses &

1. Assessing Uncertainties in Observations, Analyses & Short-Range Weather Forecasts A Proposal for Verification-Related Activities Within GSD Prepared by the FAB Implementation Plan Team Seth Gutman, Steve Albers, Dan Birkenheuer, Ed Tollerud, Yuanfu Xie, and Zoltan Toth GSD Verification Summit September 8, 2011

2. Introduction • One recommendation from 2010 ESRL Physical Science Review Panel was that “... ESRL should build a stronger effort in observing, data assimilation and forecast system evaluation and optimization.” • GSD/FAB was charged with developing a plan to: • Assess observational uncertainty estimates for DA, forecast systems & OSSEs • Improve our ability to monitor the quality of major remote sensing and in-situ observing systems • Evaluate the potential impact of improved observation error specifications on attaining NOAA NWP forecast system performance objectives. • The plan was circulated within GSD and comments from outside groups were solicited & incorporated.

3. Problem Statement • Currently, observation errors (instrument errors + representativeness errors) as well as analysis errors & forecast errors are not estimated independently. • For example, Obs errors are typically treated as “tuning parameters” in many DA systems. • This can lead to sub-optimal analyses impacting weather forecast accuracy, or erroneous conclusions from OSSEs. • To see continued improvement in short range NWP, and/or maximize the return on investment in new observing systems, alternatives to these ad hoc processes should be developed and then implemented operationally in NOAA.

4. Proposed Objectives • Developobjective, scientifically sound methods for the routine estimation of observation, analysis, and short range forecast errors. • Demonstrate the ability of such methods to continuously monitor the performance of observing, DA, and forecasting systems. • Demonstrate that improvements in analysis and forecast accuracy result from improved observation error specifications. • Determine if there is a path to transition these methods into operations, and a way to identify the resources to accomplish it.

5. Proposed Scope of Work • Form a cross-GSD Team to collect information and/or perform experiments to gain more information on: • Instrument errors. • Representativeness errors & scale dependence. • Forward model errors. • Strategies to monitor and QC observing systems.* • The value of making QC part of variational DA by fully utilizing improved observational error estimates.* • Analysis and short-range forecast errors, including the correlation between analysis and forecast errors due to cycling of forecasts in DA. • Observing System Simulation Experiments (OSSEs). *Links to possible QC activities

6. Expected Benefits • Since observational, analysis, and forecast errors are all interlinked, we anticipate: • better GSD-wide estimates of obs errors than competing (research & operational) DA and forecast systems • which will result in • improved performance of GSD DA and forecast systems • that in turn may further reduce uncertainty in the estimates of observational errors • and improve the accuracy & reliability of GSD OSSEs • Put GSD in a strong position for future domestic & international modeling activities.

7. New Business Lines? • For consideration: ESRL as an independent monitor of critical (weather & climate) remote sensing and in situ observations for NOAA and other agencies?