An introduction to data assimilation Xiang-Yu Huang Danish Meteorological Institute, Denmark

1. An introduction to data assimilation Xiang-Yu Huang Danish Meteorological Institute, Denmark

2. Outline of the presentation • Operational NWP activities • Observations and preprocessing • There are still many observations we are not able to assimilate. • We have to prepare for new observations to come. • Observation operators H • Error covariances B and R • They determine the assimilation quality. • We can only guess what they should be. • Data impact • It can take decades of hard work just to assimilate one data type. • How to assess data impact is application dependent. • Summary and our near future plan.

4. Numerical Weather Prediction: models and initial values

5. DMI-HIRLAM The operational system consists of three nested models named "G", "E" and "D".

6. Data assimilation cycles

7. SYNOP SHIP BUOY

8. AIREP AMDAR ACARS

9. TEMP PILOT

10. ATOVS

11. Comments (I):Observations alone are not enough. • Observations only cover part of the model domain (for limited area models they could also be outside of the model domain). • Some observations provide incomplete model state at given locations (e.g. only wind). • Some observations are not NWP model variables (e.g. radiance). NWP is not the only purpose of making observations.

12. Quality controlObserving systems have problems. • Bad reporting practice check • Blacklist check • Gross check (against some limits) • Background (short-range forecasts) check • “Buddy check” (against nearby observations) • Redundancy check • Analysis check: OI check or VarQC

15. Received Assimilated

16. Comments (II):We are far from using all the observations. • Data quality dependent. • Observing system dependent. • NWP model (resolution) dependent. • Assimilation method dependent. At the same time, we have to prepare for the new data like RO to come.

17. Routine monitoring Short-range forecasts - observations

18. Analysis methods • Empirical methods • Successive Correction Method (SCM) • Nudging • Physical Initialisation (PI), Latent Heat Nudging (LHN) • Statistical methods • Optimal Interpolation (OI) • 3-Dimensional VARiational data assimilation (3DVAR) • 4-Dimensional VARiational data assimilation (4DVAR) • Advanced methods • Extended Kalman Filter (EKF) • Ensemble Kalman Filter (EnFK)

19. Variational methods dx (new) (initial condition for NWP) (old forecast)

20. Algorithms

21. Important issues • H observation operator, including the tangent linear operator H and the adjoint operator HT. • M forecast model, including the tangent linear model M and adjoint model MT. • B background error covariance (NxN matrix). • R observation error covariance which includes the representative error (MxM matrix).

22. Observation operator H: from model state x to observations y This is mainly for conventional “point” observations. Horizontal and vertical integration (not interpolation) may be needed for most remote sensing data.

23. Examples of specific observation operators • For direct model variable observations, Hspec = I. • Radial winds: • Integrated water vapour: • Refractivity: • For radiance data, RTTOV-7 (a complicated software).

24. Level of preprocessing and the observation operator Hspec=HFHGHRHN Raw data: Phase and amplitude Frequency relations Ionosphere corrected observables Hspec=HGHRHN Geometry Hspec=HRHN Bending angle profiles Abel trasform or ray tracing Hspec=HN Refractivity profiles Hydrostatic equlibrium and equation of state Hspec=I Temperature profiles

25. Basic assumptions • Observations are unbiased. (Bias removed.) • Background is unbiased. (Bias removed?) • Observation error covariance matrix is known. R • Background error covariance matrix is known. B • Observation errors and background errors are not correlated.

26. Observation errors, computed for GPS/MET geopotential data (using ECMWF analyses as “TRUTH”)

27. Estimate B without “TRUTH” • The NMC method • Background error covariances are proportional to correlations of differences between 48 h and 24 h forecasts valid at the same time. • The analysis ensemble method • Several analyses are performed with perturbed observations. Differences between background fields are used to estimate background error covariances.

28. The Hollingsworth-Lönnberg method. (Estimate both B and R without “TRUTH”) B: R:

29. ZU ZV ZZ VU VV VZ UU UV UZ Horizontal multivariate correlation: spread the information

30. Wave number Vertical correlation (spread the information) for the temperature at 500 hPa Pressure (hPa)

31. Analysis increments due to 5 GPS/MET Z profiles

32. Comments (III)We need to estimate observation errors now and then. • Observation errors include representative errors. • Observation errors should be estimated for each model system. • Observation errors may need to be re-estimated for each model refinement and instrument improvement. • (It is believed that it is more important to get o /b right than to estimate o and b.)

33. Comments (IV)We need to estimate the background errors again and again. • Spread information (but could also cause “problems”) • horizontally • vertically • to other variables • Impose balances to the analysis. • Background errors should be estimated for each model system and be re-estimated for each model improvement. • (It is believed that it is more important to get o /b right than to estimate o and b.)

34. From research to operations • Development and simple checks • Coding • Analysis increments • Case studies • Extensive experiments (e.g. one month for each season) • “Standard scores”: bias, rms, correlation, etc. • Special scores: precipitation, surface fluxes, etc. • Special aspects: noise, spin-up, etc. • Pre-operational tests • Operational use (feedback to further research)

35. MSLP Z850 T850 V850 T02M Z500 T500 V500 V10M Z200 T200 V200 RH850 RH500 Observation verification against EWGLAM station list Jan 2003 NOA (No ATOVS) WIA (With ATOVS) ATOVS into DMI OPR since 2002. (A) TOVS work started in 1988 (Gustafsson and Svensson)

36. Observed and predicted (+12h) precipitation Observed Without ZTD With ZTD

37. Recent HIRLAM impact studies 1. EWP: minor positive impact; blacklisting and bias correction may be needed. 2. MODIS wind: slightly negative; obs errors, screening procedures and level assignment need to be investigated. 3. MODIS IWV: neutral obsver, but positive on heavy precip cases. 4. GPS ZTD: neutral impact on most meteorological parameters, but positive impact on heavy precipitation cases. 5. AMSU-A: positive impact for the recent two-month experiment. The firstguess check is important. 6. Quikscat: positive impact

38. Comments (V)We need to assess data impact regularly. • It can take years and decades for an observing system to reach the operational status. • An observing system in operational use may also become redundant due to advances in assimilation techniques, new observing systems and improvements in other components. • Continuous monitoring and further tuning are necessary to keep an observing system in the operational use.

39. Other important aspects • Balanced motion • Adjustment and initialisation • Flow dependent B • Non-Gaussian statistics

40. Summary • Observations alone are not enough. • We are far from using all the available observations, and at the same time we have to prepare for the new data to come. • The “statistics” is evolving: • Observational errors • Background errors • It is getting more difficult for a new observing system to have a positive impact, as • NWP models become better • Other existing observing systems become better

41. Assimilating Radio Occultation data • Global data coverage. • Good vertical resolution (in contrast to most other satellite data). • Insensitive to cloud and precipitation. • Positive impact from real data collected from a single LEO has already been found on one of the most advanced data assimilation systems. We will start soon after this workshop - next week!