Perspectives on Targeting

1. Perspectives on Targeting Sharanya J. Majumdar (RSMAS/U. Miami) Session3.1, THORPEX/DAOS WG Fourth Meeting 27-28 June 2011.

2. Introduction • 3rd DAOS WG meeting in Montreal (July 2010): reviewed entire procedure: case selection, objective techniques, evaluation methods. • This talk: focus on outstanding issues raised in review article.

3. Outline • Status of review article • Main Conclusions • Open questions and issues • Recommendations

4. Status of review article • “Targeted observations for improving numerical weather prediction: An overview” • Version 1.0 completed 12/20/10 • Version 2.0 completed 2/18/11 for WWRP/JSC • Version 2.2 sent to DAOS WG 3/21/11 • Version 3.0 completed 5/24/11, sent to all co-authors Goals • To revise Version 3.0 and finalize as a THORPEX report by 31 July • To truncate manuscript by ~50% and submit to BAMS by 31 August. How to communicate results?

5. Mid-latitude weather (NCEP GFS) • Szunyogh et al. (2000, 2002) • ~70% of all WSR cases improved • average 12-h gain in lead time • Yucheng Song (2010, pers. comm.) • ~70% of 128 cases between 2004-7 improved • ‘Large’ improvements in 5/20 cases each year • Average forecast error reduction 10-20% within verification area for NOAA’s ‘high priority’ cases • Winter T-PARC: 75% of the 52 cases were improved • Do conclusions from Szunyogh et al. (2000, 2002) still hold?

6. Mid-latitude weather • Szunyogh et al. (2002), Petersen et al. (2007), Majumdar et al. (2002, 2010): target areas often in cloudy and/or baroclinicregions. • ETKF demonstrated the ability to quantitatively predict reduction in forecast error variance, particularly in zonal flows. • Target regions traceable from Japan to North America for 4-7 day forecasts. • Other THORPEX experiments: very small sample size of specific targeting cases.

7. Mid-latitude weather: Conclusion • “In conclusion, the average cumulative value of targeted data in the mid-latitudes has been found to be positive but small.” • Possible reasons for small impact: • dropwindsondedata are • relatively few • collected in limited area that does not cover full sensitivity • not deployed continuously in time. • improved routine observational network • advancements in modeling and data assimilation combine to improve analyses and thereby limit the potential for extra observations to improve the forecast • targeting strategies are limited • operational data assimilation schemes not yet fully-flow-dependent

8. Tropical cyclones • Main metric to date: TC track. Targeted observations have proven to be useful statistically. • Benefit to society more straightforward to define than for mid-latitude weather. • An effective strategy: observe uniformly around TC • Extra observations in target areas have been shown to be useful on a case-by-case basis. • Quantitative benefit differs from model to model • 3d-Var versus 4d-Var • Treatment of routinely-available satellite versus aircraft observations • Improvements to large-scale pattern downstream? • See T-PARC talk for further details

9. General conclusions • Spatial / temporal range of aircraft is a limiting constraint. • Cumulative adjoint-based observation impact of a small number of targeted aircraft observations to forecast accuracy over broad verification regions is smaller than that of any individual observing system. • Results areheavily dependent on the flow regime, the numerical model, and the treatment of both routine and targeted observations in the data assimilation scheme. • Current assimilation methodologies are limited in their use of fully flow-dependent spatial structure functions to control the spreading of the influence of observations.

10. Targeting routinely available data • ECMWF trifecta: • Kelly et al. (2007): impact over North America from observations over the Pacific exceeds that over Europe from Atlantic observations • Buizza et al. (2007): observations taken in SV target areas were on average more valuable than those taken in randomly selected areas (winter: forecast errors reduced by 4% over Pac / 2% over Atl). • Cardinali et al. (2007): removal of observations in SV-sensitive areas degraded skill more so than observations selected randomly.

11. Targeting routinely available data • Gelaroet al. (2010) • Global impacts of the major observation types on 24 h forecast errors are similar in each system. • Largest forecast error reductions due to assimilation of satellite radiances, geostationary satellite winds, rawinsondes and commercial aircraft (similar to Langland and Baker 2004) • Only a small majority (50-54%) of the total number of observations assimilated improved the forecast

12. Targeting routinely available data • Bauer et al. (2011) • Influence of adding satellite data at 0.625° resolution in ECMWF SV-sensitive areas. • 2-day forecasts of 500 hPa Z most improved when increased data were assimilated in flow-dependent SV areas, as opposed to in randomly distributed areas or climatological SV areas. • Forecast impacts larger in SH than NH, dependent on season.

13. Medium-range: conclusions • Some studies: targeted observations made negligible differences to forecasts downstream • Others: positive impacts in mid-latitudes and tropics. • Results are not mature enough to make an authoritative statement. • Given that model error is expected to amplify at longer lead times for synoptic-scale processes, the value of targeted observations in the medium- and long-ranges is expected to diminish, particularly if the observations are confined to a local region. • Broader-scale, regime-based sampling (e.g. adaptive use of satellite data) is expected to be one promising approach to address targeting for long-range forecasts.

14. Questions: Case selection • Can low-impact cases be predicted in advance? • Low-impact weather • Expected low impact of extra obs on forecast • Is ensemble spread / ETKF signal variance / any other measure an accurate predictor of potentially useful cases for targeting? • Do we have a scientific idea of low predictability regimes in which targeted observations would likely yield significant benefit?

15. Questions: verification 1 • Large, fixed verification regions • Increase level of automation • Increase breadth of observations available for verification • Verification statistics are diluted in large areas that are not directly affected by weather event • Small, mobile verification regions • Centered on weather event that is targeted (or maximum in spread or signal variance) • Relatively few observations available for verification

16. Questions: verification 2 • Many studies evaluate average improvements, in a verification region and/or over many cases. • Better to focus on individual systems: select only high-impact, low predictability cases for regional targeting? • Is a field campaign judged a success if (say) two high-impact forecasts are substantially improved while the improvement is minimal in others? • How many forecasts per year are truly busted? • Local or global verification metrics?

17. Questions: value of targeting 1 • What types of forecast improvements are sufficient to justify a mobile observing network based on a targeting strategy? • Depends on the agency, e.g. NOAA uses aircraft recon since the impact on NCEP models is deemed satisfactory, even if there is little impact in ECMWF. • Expectation: the average improvement due to targeted data should be small.

18. Questions: value of targeting 2 • We can quantify improvement to forecast skill, but can we translate this into forecast value? • Can we evaluate cost-effectiveness and benefits to society?

19. Questions: recent progress • Significant progress since Langland (2005)? • For tropical cyclones, clearly YES • For mid-latitudes, not clear • Medium range: some progress • Advances in satellite data targeting • Adjoint-based observation impact widely used • New field campaigns: Concordiasi etc.

20. Recommendations • Would like to make ~5 concrete recommendations in paper.

21. Potential Recommendations • Quantify benefit ofadapting the network of existing regular satellite and in-situ observations in a targeted sense: • On-request rapid-scan wind data • Targeted satellite channel selection and data-thinning • Increase observations from commercial aircraft • Request radiosondes at non-standard time • Explore the utility of observations in cloudy areas • Annual field campaigns require continuous evaluation: to justify their benefit and to determine how to derive further benefit as observations, models and DA evolve. • Make assessments using the most up-to-date assimilation systems and models. • Investigate regional targeting of flow regimes with lower predictability on a continuous basis for periods of days to weeks. This may be more effective than occasional, limited-area sampling if a global verification norm is used.

22. Potential Recommendations • Investigate forecast dropouts. • Continue to develop innovative targeting techniques that distinguish between observations that significantly reduce forecast error variance and those that do not. • Evaluate whether data assimilation schemes that better account forflow dependencecan identify gaps in data sensitive regions that current data assimilation schemes fail to fill. If so, does this increase the impact of targeted observations? • Scientific studies on targeting methods in operational models; impact of targeted observations; dynamics of error growth; uncertainty prediction.