Assessing forecast uncertainty from synoptic to sub-seasonal scales.

1. Celeste Saulo and Juan Ruiz CIMA (CONICET/UBA) – DCAO (FCEN –UBA) Assessing forecast uncertainty from synoptic to sub-seasonal scales.

2. Motivation and general context • Many meteorological services run operational ensemble prediction systems (EPS), which provide estimates of the uncertainty of the forecast. • Many of these outputs are readily available to the scientific community through, e.g. TIGGE (THORPEX Interactive Grand Global Ensemble). • Obtaining useful (valuable) information from EPS requires statistical post-processing and specific research depending on the variable/problem/region. • There is growing interest in obtaining useful information from EPS on time scales between 2 weeks and 2 months.

3. Motivation and general context • Active research is being pursued in numerous places on the definition of initial ensembles, multimodel (or stochastic physics) as well as on the evaluation of ensemble predictions. • During the first half of THORPEX it was realized that model error diagnosis is one area where universities and research institutions can make substantial contributions to the further development of models (and hence forecast skill), thereby supporting the relatively small community of model developers. THORPEX = The Observing System Research and Predictability Experiment

4. Potentialareas of researchunder UMI-IFAECI • Predictability studies • Ensemble generation (including data assimilation) • Probabilistic forecasts • Verification strategies

5. Related ongoing studies • How sensitive are probabilistic precipitation forecasts to the choice of calibration algorithms and the ensemble generation method? • Part I: Sensitivity to calibration methods (Ruiz and Saulo, Meteorol. Appl., 2011) • Part II: sensitivity to ensemble generation method (Ruiz, Saulo and Kalnay, Meteorol. Appl. 2011)

6. Three different ensemble generation strategies, using WRF regional model as the basis: • Breeding (11 members) • Multi-model (11 members) • Pragmatic= spatially shifted ((2*m + 1)2 members, e.g., 121)

7. In order to correct the effect of the ensemble systematic errors, several techniques have been developed, all of them based in the study of the relationship between error and forecasted value and in the development of statistical models to compute a calibrated probability given the forecasts of the ensemble members • A logistic regression is used to represent h(y>0|f) and a GAMMA function is used to represent h(y>tr|f,y>0) • BMA → weighted + calibrated probability for each member • GAMMA-ENS →all weights are equal + calibrated probability for each member • GAMMA→ no weights + calibration applied to the ensemble mean • WMEAN →weighted ensemble mean and then calibration is applied

8. Weights associated to each member of the spatially shifted ensemble as a function of the corresponding shift in the south–north (y axis) and the west–east (x axis) directions. Negative shift values indicate southward and westward shifts respectively.

9. Continuous ranked probability score (CRPS) GAMMA calibration has beenadopted The computation of a weighted ensemble mean can lead to moderate better results; however the best choice for a weight computation algorithm is still an open question. The PQPF derived from the un-weighted ensemble mean produces, if not the best results, almost as good results as any other approach.

10. Shifted MM Combined Breeding 48 hours forecast 24 hours forecast

11. Shifted-MM Shifted-Breeding Shifted combined shifted multimodel 1331 members shifted breeding 1331 members shifted combined 2541 members

12. Precipitation uncertainty at these ranges is mostly related with the location of rain areas • The spatially shifted ensemble proves to be quite competitive at short forecast ranges, • yet its skill drops rapidly with increasing lead times uncertainties associated with the existence, or intensity of pp, tend to become more important with increasing lead times. • multimodelensemble (physics) outperformed the breeding ensemble (I&BC). Still, the improvement combining both is modest most of the PQPF limitations during summer arise from errors in model physics rather than problems in the initial and/or boundary conditions

13. Among the alternatives that have been evaluated, the most important improvement has been obtained with the combination of the multimodel ensemble approach (and/or the combined approach) and the spatial shift technique even at 48-hours lead time. This approach is particularly interesting and promising for implementing high resolution ensembles in small operational or research centers for which computational costs largely restrict ensemble size.

14. Ensemble Forecast Object Oriented Verification Method • Work in progress Juan Ruiz (postdoc at LMD) and Olivier Talagrand • The method has been designed to be applied to the 500 hPa field, however it can be easily extended to other fields as well (and probably other “objects” i.e. jet streak position, lowlevel jet maximumpossition, etc). • It is based in the identification of local minima and the system associated with each local minima. • As in 500 hPa, usually low pressure systems appear in the form of troughs rather than in the form of closed systems, the geopotential height anomaly is used instead of the full 500 hPafield.

15. Cyclone trajectories at 500 hPa, for a particular day derived from the NCEP ensemble system

16. Questionsforfutureresearch • How much information can be obtained from the ensemble spread about the forecast skill? Are there specific scores to quantify this relationship in terms that it becomes useful for particular applications? • Which is the most convenient way to combine different ensemble members? Is it necessary to take into account the different skill of each member? (i.e. Bayesian model averaging trying different weights against simpler techniques like logistic regression for precip) • Which kind of information/type of scores could be used to provide valuable information about weather states with more than two weeks in advance? • How can we use model error statistics to understand which processes are strongly affecting forecast quality so that key problems can be isolated and models improved? • Which methodologies should we apply to forecast probability of extreme events?