Local Predictability of the Performance of an Ensemble Forecast System

1. Local Predictability of the Performance of an Ensemble Forecast System Liz Satterfield and Istvan Szunyogh Texas A&M University, College Station, TX Third THORPEX International Science Symposium Monterey California, 14-18 September 2009

2. Introduction • Ensemble prediction systems account for the influence of spatio-temporal changes in predictability on forecasts • Performance of an ensemble prediction system is flow dependent • The goal of our study is to lay the theoretical foundation of a practical approach to predict spatio-temporal changes in the performance of an ensemble prediction system

3. Experiment Design • We use an implementation of the Local Ensemble Kalman Filter (LETKF) on T62L28 resolution version of the NCEP GFS • Experiments with Observations: • Simulated Observations in Random Location: 2000 randomly placed vertical soundings that provide 10% coverage of model grid points (Kuhl et al. 2007, JAS). • Simulated Observations at the Location of Conventional Observations: Observational noise added to “true states”, location and type taken from conventional observations • Conventional Observations of the Real Atmosphere: Observations used to obtain the type and location for simulated observations (excludes satellite radiances)

4. Linear Diagnostics calculated in local regions using energy rescaling • Explained Variance Fraction of forecast error contained in the space spanned by the ensemble • Minimum value of zero when the error lies orthogonal to the space spanned by ensemble perturbations • Maximum value of 1 when the ensemble correctly captures the space of uncertainty • E-Dimension A local measure of complexity based on eigenvalues of the ensemble-based error covariance matrix in the local region (Introduced in Patil et al. 2001) • Minimum value of 1 when the variance is confined to a single spatial pattern of uncertainty • Maximum value of N when the variance is evenly distributed between N independent spatial patterns of uncertainty

5. Relationship between Explained Variance, E-Dimension, and Forecast Error shown for conventional observations Colors show mean E-dimension Joint ProbabilityDistribution Linear space provides an increasingly better representation of the space of uncertainty up to 120 hours Higher Forecast Error StrongInstabilities Lower E-Dimension

6. Local Relative Nonlinearity a measure of linearity in the local regions Distance between ensemble mean and control forecasts normalized by the average perturbation magnitude Time mean of globally averaged values for conventional observations Local Regions Global  = || xa,f-xa,f|| / ___(1/k)||xa,f(k)|| Standard deviations of values computed using localization show a high degree of variability Modified from Gilmour et al (2001) Forecast Lead Time

7. Correlation between relative nonlinearity and explained varianceshown for conventional observations High values of explained variance at the 120 hour lead time are not due to strong linearity of the evolution of uncertainties

8. Evolution of Forecast Error shown for randomly placed simulated observations For a perfect ensemble, TVand TVswould equal Vsat the initial time TV = Square of the magnitude of the error in the ensemble mean forecast TVs =Portion of TV which lies in the space spanned by the ensemble perturbations Vs = ensemble variance At initial time, TVs equals Vs therefore further inflating the variance would not improve analyses Forecast Lead Time

9. Evolution of Forecast Error results shown for the Northern Hemisphere Extratropics Simulated obsRealsitic location Conventionalobs. TV TVs Vs Forecast Lead Time Forecast Lead Time The total ensemble variance underestimates the forecast error captured by the ensemble

10. Spectrum of the Ratio Between Observed and Predicted Probability (d-ratio) at analysis time Modified from Ott et al (2002) Simulated obsRealsitic location dk=(xtk)2/k Optimal performance in this measure would be indicated by 1 for all k Simulated obsRandom location Conventionalobs. Uncertainty is underestimated Uncertainty is overestimated eigen-direction

11. Spectrum of the Ratio Between Observed and Predicted Probability (d-ratio) at 120-hour lead time Simulated obsRealsitic location di < 1 : ensemble overestimates di > 1 : ensemble underestimates Simulated obsRandom location The spectrum is steepest for observations of the real atmosphere Conventional obs eigen-direction By 120-hr lead time, the ensemble underestimates uncertainty in all directions

12. The leading direction of d-ratio calculated for temperature at 850hPa Obs. of the real atmosphere Simulated, Realistically Placed For realistically placed observations, the regions of largest underestimation are those of highest observation density

13. Conclusions • The linear space spanned by ensemble perturbations provides an increasingly better representation of the space of uncertainties with increasing forecast time. • The improving performance of the space of ensemble perturbations with increasing forecast time is not due to local linear error growth, but rather to nonlinearly evolving forecast errors that have a growing projection on the linear space. • At analysis time, we find that the ensemble typically underestimates uncertainty more severely in regions of high observation density than for regions of low observation density. This result indicates that implementing a spatially varying adaptive covariance inflation technique may improve analyses.