Specification and estimation of observation errors in reanalysis

1. Specification and estimation of observation errors in reanalysis Paul Poli Acknowledgments to Hans Hersbach, who converted observation feedback from Compo et al. 20th Century Reanalysis into ODB ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

2. Outline • Initial considerations about errors and scope of this talk: observation error standard deviation (sigO) • Review of observation errors (sigO) • used in ERA-Interim/ERA-40 • Application of Desroziers’ method • Surface pressure observations from 20CR reanalysis • Observations from ERA-Interim ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

3. Overly-simplistic view of error Some N-dimensional geophysical space Example of a state: the average temperature in Vienna city center (30 km sq. area) between 19 April 2012 12:16 UTC and 12:30 UTC is 16 degrees C between 1.5 and 2.5 meters above ground. Error Estimated state True state If we make further estimations (or take more measurements) Some N-dimensional geophysical space True state Distribution of estimated states ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

4. Distribution of errors Count Some N-dimensional geophysical space True state Distribution of estimated states ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

5. About these distributions X-axis and values Y-axis and values How do you count? What is the shape of the distribution? How do you qualify the center of the distribution? More often than not: median <> average <> mode With how many parameters can you model this distribution? E.g. zero-mean Gaussian  only stdev. Is there a change over time? Stationary process often assumed • What are the relevant geophysical dimensions? • Along which dimension do you estimate the error? • Does it depend on the actual state? Age of the instrument? … • Where is the zero? • Reference calibration, maybe using low values (e.g. cosmic background) • How do you scale each dimension? • Is there some systematic scaling e.g. against a controlled “hot” blackbody (or some radio wavelength controlled by an atomic clock) 0 0 ? ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

6. What this talk WILL and will NOT cover X-axis and values Y-axis and values How do you count? What is the shape of the distribution? How do you qualify the center of the distribution? More often than not: median <> average <> mode With how many parameters can you model this distribution? E.g. zero-mean Gaussian  onlystdev. Is there a change over time? Stationary process often assumed Gaussian • What are the relevant geophysical dimensions? • Along which dimension do you estimate the error? • Does it depend on the actual state? Age of the instrument? … • Where is the zero? • Reference calibration, maybe using low values (e.g. cosmic background) • How do you scale each dimension? • Is there some systematic scaling e.g. against a controlled “hot” blackbody (or some radio wavelength controlled by an atomic clock) Zero 0 0 ? ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

7. Hereafter, consider only sigO = observation error standard deviation • We thus ignore: • Systematic errors, or biases. Not treated here explicitly, although these biases are ubiquitously present. • Correlation of errors between observations. Same caveat applies. They are always present and SHOULD not be ignored. ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

8. Summary of the observing system (1/3) Barometers (sometimes using temperature readings to bring back to sea-level) Thermometers Hygrometer, or psychrometer (dry and wet-bulb thermometers) Inference of drift caused by atmospheric motion Anemometers ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

9. Summary of the observing system (2/3) u/v components of wind, from satellite imagery Bending of GPS radio waves u/v components of wind, from backscatter Ozone layer content, from satellite (various techniques) ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

10. Summary of the observing system (3/3) Radiance counts, converted to brightness temperature Each bar represents one instrument flown on one satellite. ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

11. Satellite radiances cover various parts of the spectrum: example: All IR instruments supported by RTTOV as of 2009 ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

12. Review of current practice in ERA (-40, -Interim) • sigO was as specified by ECMWF Operations at the time each reanalysis system was built (2002, 2006) • As such they rather describe a MODERN observing system • None of these have any time-variation ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

13. In the rest of this talk, we will look at • Two kinds of sigO • As specified in the assimilation in ERA-40, ERA-Interim • As estimated by Desroziers’ method • To avoid coming to completely phony conclusions, these metrics should always be looked at along with the following quantities: data count, O-B, and O-A. • Mostly, time-series • Enables to gauge whether our stationary assumptions are valid • Captures many other aspects of the estimation process ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

14. Newly acquired dataset: International Surface Pressure Databank v2.2 (ISPD) • Used by Gil Compo’s 20CR reanalysis • Contains • Observation minus background • Observation minus analysis • From this, we compute sigO_Desroziers • <(O-A)*(O-B)>=<e0**2> ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

15. Dimensions of the dataset • Date, time • Location • Station and collection • Report type: from fixed station over land, or from (moving) ship or buoy • Pressure reported at sea-level, or at station-level • We will look at the influence of each of the above ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

16. sigO_Desroziers from 20CR:sea-level reporting vs. station-level reporting (hPa) 1900 2010 3 Sea-level reporting 1 Station reporting 600k Station reporting Sea-level reporting Nobs/month ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

17. sigO_Desroziers from 20CR: Influence of station altitude (hPa) 1 2 3 4 Nobs Station reporting Station altitude (m) Sea-level reporting Nobs In ERA-40/ERA-Interim, as in NWP: increasing altitude-dependency for errors at station-level ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

18. sigO_Desroziers from 20CR / ERA-20C prototype: Influence of station altitude Sep-Oct 1901 Assumptions in ERA Station reporting Mean O-B Station altitude (m) 20CR sigO ERA-20CsigO 20CRstdv O-B ERA-20Cstdv O-B Sea-level reporting 0 1 2 3 4 Nobs (hPa) ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

19. sigO_Desroziers from 20CR:Influence of report type (hPa) Ship, reporting at sea-level 3 Buoy, reporting at sea-level 2 0.85 1 Station, reporting at sea-level Station, reporting at station-level 0.7 0.6 1900 2010 In ERA-40/ERA-Interim ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

20. sigO_Desroziers from ERA-Interim: Ps from Stations with personnel ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

21. sigO_Desroziers from ERA-Interim: Ps from Automated Stations ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

22. sigO_Desroziers from ERA-Interim: Ps from Ship ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

23. sigO_Desroziersfrom 20CR & ERA-Interim (hPa) Ship, reporting at sea-level 3 2 from ERA-Interim, for Ship 0.85 1 1900 2010 In ERA-40/ERA-Interim ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

24. sigO_Desroziersfrom 20CR & ERA-Interim (hPa) 3 2 from ERA-Interim, for Manual Land SYNOP 1 Station, reporting at sea-level Station, reporting at station-level 0.7 0.6 1900 2010 In ERA-40/ERA-Interim ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

25. sigO/sigB/sigADesroziers applied to Radiosonde Temperatures sigA Dec 2011 sigB Dec 2011 sigB Dec 1979 sigA Dec 1979 sigO Dec 2011 10 hPa sigO Dec 1979 100 hPa 1000 hPa Radiosonde equipment has changed quite a bit in 30 years… Probably shouldn’t assumed errors constant in time or across RS types ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

26. sigO_Desroziers from ERA-Interim: Radiosonde Temperatures 1000 hPa 500 hPa 100 hPa 50 hPa ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

27. sigO_Desroziers from ERA-Interim: radiosonde winds ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

28. sigO_Desroziers from ERA-Interim: HIRS Channel 4 Constant, and independent of the satellite ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

29. sigO_Desroziers from ERA-Interim: HIRS Channel 4 (14.22 microns) (Now) known problem with NOAA-18 HIRS (loose lens) Interesting to see how each estimate seems to be affected by the introduction of another similar instrument… ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

30. sigO_Desroziers from ERA-Interim: AIRS Ch. 186 (14.23 microns) and 190 (14.21 microns) ??? Scale of variations quite small ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

31. sigO_Desroziers from ERA-Interim for GPS radio occultation bending angles 8-12 km impact height Fairly constant overall ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

32. sigO_Desroziers from ERA-Interim for GPS radio occultation bending angles Separated by instrument, 12-16 km Estimated Used ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012

33. Conclusions • sigO assumed in ERA-Interim/ERA-40 constant over time • Estimates using Desroziers’ method suggest reasonable broad features: • E.g. large improvements over time in components of the observing system known to have improved such as radiosondes • Stable instruments (AIRS, GPSRO) show overall stable errors • We spotted an error in how Ps obs. stdv. errors are specified in our system for observations reported at station level. Probably a remnant of old days when it was thought that these should be associated with higher representativeness errors? • Some other features, however, are puzzling • E.g. jumps in estimated sigO when instruments are introduced/removed … effect of correlations? • The estimates differ quite a bit depending on the reanalysis: • E.g. 20CR (ensemble mean) vs. ERA-Interim (deterministic 4DVAR) give different results for Ps ERA-CLIM Workshop on obs errors, U.Vienna, Apr 2012