Systematic Errors (Part I)

1. Systematic Errors (Part I) Thomas Jung European Centre for Medium-Range Weather Forecasts

2. Scope of Part I and II Question: “Do we still have significant systematic errors in the ECMWF forecasting system?” • If so, what are the main problems? • How did systematic errors evolve over the years? • How do systematic errors grow? • How well do we simulate specific phenomena (e.g., blocking, extratropical cyclones)? • Does increasing resolution help? • Which techniques can be used to understand systematic error?

3. Two kinds of forecast error • Random error (model+initial error) • Systematic error (model error*) Introduction Two principal sources of forecast error: • Uncertainties in the initial conditions (“observational error”) • Model error

4. Concept of Systematic Error • Relatively straightforward to compute • BUT can be difficult to identify the origin of errors • Moreover there are pitfalls: • finite length (significance tests help partially) • apparent systematic error for short time series (loss of predictability) • Observations might be biased

5. Observed Anomaly “Systematic Error” when predictability is lost Fingerprint of loss of predictability: “Systematic error” resembles the opposite of the observed anomaly Loss of Predictability and “Systematic Error” “Forecast” “Observed”

6. Observed Anomaly Mean Error @ D+10 Example: Z500 DJF 2005/06 Spatial correlation=-0.78

7. Uncertainty of our “Truth”

8. Data • Systematic errors and their growth • Operational forecasts (to D+10) • ERA-40 reforecasts (to D+10) • EPS control forecasts (to D+20) • Monthly forecasts (to D+30) • Seasonal forecasts (beyond D+30) • Evolution of systematic errors • Operational forecasts + analyses • Verification (ERA-40, observational products)

9. D+1 D+3 D+5 D+10 Systematic Z500 Error Growth from D+1 to D+10

10. Asymptotic Z500 Errors (DJFM 1962-2001)

11. Asymptotic Z500 Errors (DJFM 1962-2001) Cycle 26r1 Cycle 26r3

12. Two Different Aerosol Climatologies

13. Systematic Error Growth: EPSC Z500 2000-2003 2003-2006

14. 1983-1987 1993-1997 2003-2007 Evolution of D+3 Systematic Z500 Errors

15. Reduction of Systematic Errors (1981-2006)

16. Systematic D+3 Z500 Errors: 3 NWP Models ECMWF Meteo-France DWD

17. Systematic Errors: AMIP Models

18. Demeter Models (DJF Precipitation) LODYC CERFACS ECMWF Meteo-France MetOffice MPI

19. Cycle 32R3: A Major New Model Cycle • Modification to the convection scheme (removal of large-scale control of convection (through ω-field) • Reduction of vertical diffusion

20. 32R2 32R3 Precipitation (DJFM 32R2 vs 32R3)

21. Precip and Velocity Potential (DJFM 32R3)

22. Remainder of Part I A selection of other key problems

23. Zonal Mean Zonal Wind

24. Synoptic Activity: D+5 to D+10 (DJF) 2001-06 2007-08

25. Synoptic Z500 Activity (32R3 DJF 1990-2005)

26. Blocking Methodology L H L

27. ERA-40 D+1 D+4 D+7 D+10 Blocking Frequency Biases (23r4) DJFM 1960-2001

28. 32R2 32R3 Asymptotic Systematic Errors

29. Schematic of the MJO From Madden and Julian (1994)

30. Near Global Impact of the MJO (Precipitation) Conv.: Indian Ocean Conv.: Maritime Continent Conv.: Central Pacific Conv.: WH/Africa

31. MJO Teleconnections: ERA-40

32. MJO Teleconnections: Cycle 32R3

33. The Madden and Julian Oscillation ERA-40 ECMWF Model

34. Conclusions (1) Main systematic errors: • Concept of systematic error very straightforward • But there are pitfalls: • Short time series (sampling issues + loss predictability). • Uncertainty about the true state of the atmosphere. • Systematic error are a clear sign of model error • Understanding systematic error is challenging (see Part II)

35. Conclusions (2) Main systematic errors: Do we still have systematic errors in the ECWMF model? • Yes, we do. • However, most systematic errors have been substantially reduced in recent years • Some key-challenges remain:

36. Atmospheric circulation over the North Pacific • Synoptic activity • Euro-Atlantic blocking • Tropical circulation + hydrological cycle • Polar vortex in the stratosphere • Madden-and-Julian Oscillation • Quasi-Biennial Oscillation • Others Conclusions (3) Main systematic errors: Key-challenges:

37. Further Reading Bechtold, P., M Koehler, T. Jung, M. Leutbecher, M. Rodwell and F. Vitart, 2008: Advances in simulating atmospheric variability with IFS cycle 32R3. ECMWF Newsletter No. 114, 29-38. Avaiable from http://www.ecmwf.int/publications/newsletters/ Jung, T. and A.M. Tompkins, 2003: Systematic Errors in the ECMWF Forecasting System. ECMWF Technicial Memorandum 422. http://www.ecmwf.int/publications/library/do/references/list/14 Jung, T., A.M. Tompkins, and R.J. Rodwell, 2004: Systematic Errors in the ECMWF Forecasting System. ECMWF Newsletter, 100, 14-24. Jung, A.M. Tompkins, and R.J. Rodwell, 2005: Some Aspects of Systematic Errors in the ECMWF Model. Atmos. Sci. Lett., 6, 133-139. Jung, T., T.N. Palmer and G.J. Shutts, 2005: Influence of a stochastic parameterization on the frequency of occurrence of North Pacific weather regimes in the ECMWF model. Geophys. Res. Lett., 32, doi:10.1029/2005GL024248, L23811. Jung, T., 2005: Systematic Errors of the Atmospheric Circulation in the ECMWF Model. Quart. J. Roy. Meteor. Soc., 131, 1045-1073. Jung, T., S.K. Gulev, I. Rudeva and V. Soloviov, 2006: Sensitivity of extratropical cyclone characteristics to horizontal resolution in the ECWMF model. Quart J. Roy. Meteor. Soc., 132, 1839-1857. Gates and coauthors, 1999: An Overview of the Results of the Atmospheric Intercomparison Project (AMIP I). Bull. Amer. Meteor. Soc., 80, 29-55. Hoskins, B.J. and K.I. Hodges, 2002: New Perspectives on the Northern Winter Storm Tracks. J. Atmos. Sci., 59, 1041-1061. Koehler, M., 2005: ECMWF Technicial Memorandum 422. http://www.ecmwf.int/publications/library/do/references/list/14 Palmer, T.N., G. Shutts, R. Hagedorn, F. Doblas-Reyes, T. Jung, and M. Leutbecher, 2005: Representing Model Uncertainty in Weather and Climate Prediction. Ann. Rev. Earth. Planet Sci., 33, 163-193.