The use of ECMWF ensemble and lagged deterministic forecasts for 3-30 day outlooks in Sweden

1. The use of ECMWF ensemble and lagged deterministic forecasts for 3-30 day outlooks in Sweden • Monthly instead of seasonal forecasting • The used of lagged forecasts (as a complement to the EPS) • Problems with weighting together different forecast systems For details see :http://www.ecmwf.int/newsevents/meetings/ forecast_products_user/Presentations2006/index.htm ECMWF User Meeting 14-16 June 2006

2. The seasonal forecasts • Not used, partly because the forecasts seem to repeat themselves ECMWF User Meeting 14-16 June 2006

3. The last four years’ ECMWF summer forecasts (issued in April) Warmer than normal Warmer than normal 2003 2004 Not warmer than normal Colder than normal 2006 2005 Warmer than normal Warmer than normal Colder than normal Colder than normal ECMWF User Meeting 14-16 June 2006

4. 2. The monthly forecast Used and found skilful, but tendencies of “jumpiness” in transitional periods ECMWF User Meeting 14-16 June 2006

5. Max +5 to 10 ”Jumpiness” experienced at a specific location clim 850 hPa temperature plume for Norrköping, southern Sweden Max +10 to 15 clim ECMWF User Meeting 14-16 June 2006

6. 3. The 21 day forecast (+9 days) Using the last three days 21d forecasts enables us to inder the trends beyond day 10, even beyond day 15 For details see :http://www.ecmwf.int/newsevents/meetings/ forecast_products_user/Presentations2006/index.htm ECMWF User Meeting 14-16 June 2006

7. Main method since summer 2003 temperature Last days ECMWF fc and EPS Last days Control +21 d forecasts ECMWF monthly forecast normal statistics 30 d 10 d 20 d ECMWF User Meeting 14-16 June 2006

8. 4. The problem of verification For details see :http://www.ecmwf.int/newsevents/meetings/ forecast_products_user/Presentations2006/index.htm ECMWF User Meeting 14-16 June 2006

9. Introduction of more ECMWF data Level of useful forecasts ECMWF User Meeting 14-16 June 2006

10. What to do? Two ways to go: • Political (cover up, play illusionist tricks or change the norms) • Scientific (go to the roots of the problem) ECMWF User Meeting 14-16 June 2006

11. Political trick: Selective sampling “I just happen to have some fresh verifications here, depicting the results during the first half of this year...” ACC=98% Slope=0.8 ECMWF User Meeting 14-16 June 2006

12. But it didn’t look that bad…. Anomaly correlation of monthly forecast for Stockholm (2 m temperature) Verif Prog More ECMWF input ECMWF User Meeting 14-16 June 2006

13. Scientific approach: • The conventional verification disregarded three factors: • Variable range of variation between 2002 and 2003 • More than one verification method should be used • Twelve forecasts per year is a too small sample For details see :http://www.ecmwf.int/newsevents/meetings/ forecast_products_user/Presentations2006/index.htm ECMWF User Meeting 14-16 June 2006

14. 2002 Higher correlation Larger errors 2003 Lower correlation Smaller errors ECMWF User Meeting 14-16 June 2006

15. Another verification method RMSE Introduction of more ECMWF data reduced the errors! MABSE ECMWF User Meeting 14-16 June 2006

16. (Lagged) verification over 24 months compared to over 12 months 12 months Verifying two years at a time 24 months ECMWF User Meeting 14-16 June 2006

17. 5. Swedish concerns about the quality of the centre’s EPS 1. Forecasters at SMHI and the Air Force do not find much use of the deterministic EPS compared to an elaborate use of the deterministic model 2. The scientists at SMHI and the MISU (Univ. Stockholm) are critical about the perturbations + (recently) the stochastic physics 3. My impression is not that the EPS is bad or has become worse, but has had problems to keep pace with the improvements of the deterministic model ECMWF User Meeting 14-16 June 2006

18. The size of the T42 EPS perturbations is very large The picture depicts the status before 1 February 2006. Since then the resolution of the deterministic system has increased by 50%, but the EPS perturbations which remain at their 1995 level of T42 42 255 511 159 ECMWF User Meeting 14-16 June 2006

19. Before 2001 there was little quality difference between perturbed and non-perturbed forecasts, amounting beyond D+5 to an ACC difference. Since then it has increased to 10-15% Difference in ACC between the unperturbed Control and a randomly selected EPS member ? ECMWF User Meeting 14-16 June 2006

20. Over spreading in during the first 24-48 hours made it difficult to use the EPS as BC for the HIRLAM • In cases of extreme or interesting events the signals often come 1-2 earlier in the T799 lagged system • In cases of consistent and skilful T799 performance the EPS keep the forecaster uncertain too long • For more details see presentation at the OD Workshop November 2005 ECMWF User Meeting 14-16 June 2006

21. The RMSE of individual EPS members The 2 m temperature forecasts for London Feb-April 2006 Unperturbed Control perturbed members ”Lagged” 1 day EPS mean ECMWF User Meeting 14-16 June 2006

22. RMSE(pert member)= 1.414 (=sqrt2) RMSE (ensemble mean) Perturbed member Control climate “Tim Palmer’s Law” Ensemble mean Figure 2.1: Schematic image of the RMS error of the ensemble members, ensemble mean, and control forecast as a function of lead-time. The asymptotic predictability range is defined as the average difference between two randomly chosen atmospheric states. In a perfect ensemble system the RMS error of the ensemble members is a factor larger than the RMS error of the ensemble mean.Courtesy, L. Bengtsson, MISU ECMWF User Meeting 14-16 June 2006

23. 6. Use of the last T799 runs forming lagged ensembles (work under development) ECMWF User Meeting 14-16 June 2006

24. EPS Mean and “Lagged Mean” 24 March 00 UTC + 84h ECMWF User Meeting 14-16 June 2006

25. EPS Mean and lagged ECMWF T799 4 April 00 UTC + 132h Valid Sun 9 April 12 UTC ECMWF User Meeting 14-16 June 2006

26. 7. The public 6-10 day forecasts Once a week, “four out of five forecasts verify” ECMWF User Meeting 14-16 June 2006

27. 6-10 day forecast presented on TV 26 January 2006 ECMWF User Meeting 14-16 June 2006

28. ECMWF User Meeting 14-16 June 2006

29. Epsogram for Stockholm ECMWF User Meeting 14-16 June 2006

30. 8. When does it pay to weight together forecasts? E1< E2 For details see :http://www.ecmwf.int/newsevents/meetings/ forecast_products_user/Presentations2006/index.htm ECMWF User Meeting 14-16 June 2006

31. 9. How should x and y, the weights, be calculated taking the forecast error correlation into account? E1< E2 For details see :http://www.ecmwf.int/newsevents/meetings/ forecast_products_user/Presentations2006/index.htm ECMWF User Meeting 14-16 June 2006

32. Certain combinations of forecasts will not yield an improved weighted mean E1< E2 ECMWF User Meeting 14-16 June 2006

33. Extension to three models?? Hypothetical error correlations 100% If all three models have the same error magnitude and correlation then the weights are 33.3% But if the errors of T799 and T399 are more correlated than the errors of T799 (T399) versus UKMO the UKMO should be weighted the most T799 vs T399 50% T799(T399) vs UKMO or an arbitrary eps-member 0% D+0 D+15 ECMWF User Meeting 14-16 June 2006

34. 10. Future challenges Extending the monthly forecasts by including precipitation and provide forecasts separately form week1, week2 and week34 - and much more…. ECMWF User Meeting 14-16 June 2006