1 / 40

Systematic Errors

Systematic Errors. Thomas Jung European Centre for Medium-Range Weather Forecasts. Scope of the Lecture. Question: “Do we still have significant systematic errors in the ECMWF forecasting system?”. If so, what are the main problems? How do systematic errors grow?

armina
Download Presentation

Systematic Errors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Systematic Errors Thomas Jung European Centre for Medium-Range Weather Forecasts

  2. Scope of the Lecture Question: “Do we still have significant systematic errors in the ECMWF forecasting system?” • If so, what are the main problems? • How do systematic errors grow? • How did systematic errors evolve over the years? • How well do we simulate specific phenomena (e.g., blocking, extratropical cyclones)? • Does increasing resolution help? • Which techniques can be used to understand systematic error?

  3. Introduction Two principal sources of forecast error: • Uncertainties in the initial conditions (“observational error”) • Model error Two kinds of forecast error • Random error (model+initial error) • Systematic error (model error*)

  4. Concept of Systematic Error • Relatively straightforward to compute • But there are pitfalls: • finite length (significance tests may help) • Systematic error may be misleading for short time series (loss of predictability) • observations might be biased • Systematic errors can be difficult to interpret.

  5. Observed Anomaly “Systematic Error” when predictability is lost Fingerprint of loss of predictability: “Systematic error” resembles the opposite of the observed anomaly Loss of Predictability and “Systematic Error” “Forecast” “Observed”

  6. Observed Anomaly Mean Error @ D+10 Example: Z500 DJF 2005/06 Spatial correlation=-0.78

  7. Do we know the true state?

  8. Data • Medium-range forecasts • Primarily hindcasts from ERA-40 and ERA-Interim (robust statistics). • Seasonal integrations: • 13 month long integrations started on 1st November of each of the years 1962-2005 (or a subset of this period). • Diagnosis of standard seasons DJF, MAM, JJA, SON (errors had at least 1 month to grow, asymptotic errors). • Most experiments at TL159 with 91 levels in the vertical. • Observed lower boundary conditions (uncoupled integrations). • Observational data: • ERA-40 • Other “observational” data sets.

  9. Systematic Error Growth How do systematic errors grow throughout the forecast?

  10. Systematic Z500 Error Growth from D+1 to D+10 D+1 D+3 D+5 D+10

  11. Systematic Z500 Errors: Medium-Range and Beyond Asymptotic: 31R2 D+10 ERA-Interim

  12. Evolution of Systematic Error How did systematic errors evolve throughout the years?

  13. Evolution of D+3 Systematic Z500 Errors 1981-1985 1993-1997 2006-2010

  14. Evolution of Systematic Z500 Errors: Model Climate 35R1 33R1 32R3 32R2 32R1 31R1 30R1 29R2

  15. Systematic Z500 Errors: Impact of Recent Changes CNT-Reanalysis (33R1) Old Convection-CNT Old TOFD-CNT Old Vertical Diff-CNT Old Radiation-CNT Old Soil Hydrology-CNT

  16. Phenomena

  17. Blocking Methodology L H L

  18. ERA-40 D+1 D+4 D+7 D+10 Blocking Frequency Errors (23r4) DJF 1990-2001

  19. ERA-40 D+1 D+4 D+7 D+10 Blocking Frequency Errors (31R2) DJF 1990-2001

  20. ERA40 33R1 32R3 29R2-32R2 Blocking Frequencies: DJF 1990-2005

  21. Blocking Frequencies: DJFM

  22. Extratropical Synoptic Systems • Two approaches: • Compute standard deviation of highpass or bandpass filtered time series (“Eulerian” approach). • Track individual systems (“Lagrangian” approach).

  23. “Observed” Tracks of Long-lived Cyclones >2 days

  24. Sensitivity to Horizontal Resolution: 29R1

  25. Sensitivity to Horizontal Resolution: 31R1

  26. Number of Extratropical Cyclones (DJFM) Lifetime > 1day

  27. Synoptic Activity Error: Evolution 35R1 32R2 33R1 31R1 32R3 30R1

  28. Resolution and Computational Effort • 33 times for TL511 • 520 times for TL1279 • 2136 times for TL2047 Increased cost compared to TL159:

  29. The Madden-and Julian Oscillation

  30. Schematic of the MJO From Madden and Julian (1994)

  31. Near Global Impact of the MJO (Precipitation) Conv.: Indian Ocean Conv.: Maritime Continent Conv.: Central Pacific Conv.: WH/Africa

  32. The Madden and Julian Oscillation Reanalysis T159 T511 T1279 T2047

  33. Conclusions (1) Main systematic errors: • Concept of systematic error is very straightforward. • But there are pitfalls: • Short time series (sampling issues + loss predictability). • True state of the atmosphere not always well known. • Systematic error are (usually) a clear sign of model error. • Understanding systematic error is challenging.

  34. Conclusions (2) Main systematic errors: Do we still have systematic errors in the ECWMF model? • Yes, we do. • However, most systematic errors have been substantially reduced in recent years. • Some key-challenges remain!

  35. Conclusions (3) Main systematic errors: Key-challenges: • Tropical circulation + hydrological cycle • Madden-Julian Oscillation • Indian Summer Monsoon • Quasi-Biennial Oscillation • Euro-Atlantic blocking • Severe weather (e.g. wind storms) • Others (surface related fields)

  36. Further Reading Bechtold, P., M Koehler, T. Jung, M. Leutbecher, M. Rodwell and F. Vitart, 2008: Advances in simulating atmospheric variability with the ECMWF model: From synoptic to decadal time scales. Quart. J. Roy. Meteor. Soc., 134, 1337-1351. Jung, T. and A.M. Tompkins, 2003: Systematic Errors in the ECMWF Forecasting System. ECMWF Technicial Memorandum 422. http://www.ecmwf.int/publications/library/do/references/list/14 Jung, T., A.M. Tompkins, and R.J. Rodwell, 2004: Systematic errors in the ECMWF forecasting system. ECMWF Newsletter, 100, 14-24. Jung, A.M. Tompkins, and R.J. Rodwell, 2005: Some Aspects of Systematic Errors in the ECMWF Model. Atmos. Sci. Lett., 6, 133-139. Jung, T, et al., 2010: The ECMWF model climate: Recent progress through improved physical parametrizations. Quart. J. Roy. Meteor. Soc, 136, 1145-1160. Jung, T., 2005: Systematic errors of the atmospheric circulation in the ECMWF model. Quart. J. Roy. Meteor. Soc., 131, 1045-1073. Jung, T., S.K. Gulev, I. Rudeva and V. Soloviov, 2006: Sensitivity of extratropical cyclone characteristics to horizontal resolution in the ECWMF model. Quart J. Roy. Meteor. Soc., 132, 1839-1857. Hoskins, B.J. and K.I. Hodges, 2002: New perspectives on the Northern winter storm tracks. J. Atmos. Sci., 59, 1041-1061. Palmer, T.N., G. Shutts, R. Hagedorn, F. Doblas-Reyes, T. Jung, and M. Leutbecher, 2005: Representing model uncertainty in weather and climate prediction. Ann. Rev. Earth. Planet Sci., 33, 163-193. Rodwell, MJ and T Jung, 2008: Understanding the local and global impacts of model physics changes: An aerosol example. Quart. J. Roy. Meteor. Soc., 134, 1479-1497.

  37. Recent Model Changes

  38. Sensitivity to Model Formulation

  39. Systematic Errors: AMIP Models

  40. Demeter Models (DJF Precipitation) LODYC CERFACS ECMWF Meteo-France MetOffice MPI

More Related