1 / 44

The Operational Data Assimilation System

The Operational Data Assimilation System. Lars Isaksen, Data Assimilation, ECMWF. Overview of the operational data assimilation cycle Computational issues Observations used by the ECMWF Assimilation System Multi-incremental 4D-Var Why is 4D-Var performing better than 3D-Var?

martha
Download Presentation

The Operational Data Assimilation System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Operational Data Assimilation System Lars Isaksen, Data Assimilation, ECMWF • Overview of the operational data assimilation cycle • Computational issues • Observations used by the ECMWF Assimilation System • Multi-incremental 4D-Var • Why is 4D-Var performing better than 3D-Var? • Recent improvements of ECMWF’s Assimilation System • Near future data assimilation implementations Content of talk

  2. Data assimilation system • The observations are used to correct errors in the short forecast from the previous analysis time. • Every 12 hours we assimilate 7 – 9,000,000 observations to correct the 80,000,000 variables that define the model’s virtual atmosphere. • This is done by a careful 4-dimensional interpolation in space and time of the available observations; this operation takes as much computer power as the 10-day forecast.

  3. The operational configuration at ECMWF • Configuration: • Deterministic model: T1279L91 (~16km) • Outer loop of 4D-Var T1279L91 and inner loops T159/T255/T255 (~125km/80km/80km) • EPS target resolution T639L62 (to 10 days) and T319L62 thereafter • Wave model (25km and 36 directions) • Implemented in operations 26 January 2010

  4. Observation pre-processing for 0000UTC main cycle Extract data for 12h period 2100-0900UTC 70sec (min. 8x1PEs) Pre-process satellite data. Cloud clearing. Scatterometer winds. 340sec (min. 16x1PEs)

  5. Analysis and forecast for 0000UTC main cycle BUFR to ODB. 200sec 4x(8-16PEs) Fetch background forecast 275sec 2x(1PE) 430s 1110s 260s 880s Analysis: trajectory, minimization and update 4320sec (3072PEs) 270s 820s 490s Surface analysis. 1010sec 4x(1PEs) 10 day forecast. 1440 t-steps 3070sec (3072PEs) (or 15h fc for cycling: 260sec)

  6. The ECMWF operational schedule

  7. 06:35 Disseminate Disseminate Disseminate Disseminate Disseminate 15:30 14:00 02:00 03:30 12h 4D-Var, obs 21-09Z 12h 4D-Var, obs 09-21Z 16:00 04:00 Operational schedule Early delivery suite introduced June 2004 06 UTC analysis 18 UTC analysis 3hFC 6h 4D-Var 9-15Z 6h 4D-Var 21-03Z 3hFC 16:40 15:40 04:40 03:40 12 UTC analysis (DA) 00 UTC analysis (DA) 18:05 T1279 10 day forecast T1279 10 day forecast 06:05 51*T639/T399 EPS forec. 51*T639/T399 EPS forecasts 17:00 05:00

  8. Data extraction Blacklisting • Data skipped due to systematic bad performance or due to different considerations (e.g. data being assessed in passive mode) • Departures and flags available for further assessment • Check out duplicate reports • Ship tracks check • Hydrostatic check Thinning • Some data is not used to avoid over-sampling and correlated errors • Departures and flags are still calculated for further assessment Model/4D-Var dependent QC • First guess based rejections • VarQC rejections Used data  Increments Analysis

  9. Conventionalobservations used SYNOP/METAR/SHIP: DRIBU: MSL Pressure, Wind-10m MSL Pressure, 10m-wind, 2m-Rel.Hum. Radiosonde balloons (TEMP): PILOT/Profilers: Wind Wind, Temperature, Spec. Humidity Note: We only use a limited number of the observed variables; especially over land. Aircraft: Wind, Temperature

  10. Satellite data sources used in the operational ECMWF analysis 13 Sounders: NOAA AMSU-A/B, HIRS, AIRS, IASI, MHS 5 imagers: 3xSSM/I, AMSR-E, TMI 3 Scatterometer sea winds: ERS, ASCAT, QuikSCAT Geostationary, 4 IR and 5 winds 4 ozone 2 Polar, winds: MODIS 6 GPS radio occultation

  11. Significant increase in number of observations assimilated Conventional and satellite data assimilated at ECMWF 1996-2010

  12. The observation operator • Observations are not made at model grid points • Satellites often measure radiances, NOT temperature and humidity • We calculate a model radiance estimate of the observation to enable comparison. • This is done with the ‘observation operator’ H. • H may be a simple interpolation from model grid to observation location • H may possibly perform additional complex transformations of model variables to ‘radiance space’ for satellite data. Model T,u,v,q Model radiance Observation Satellite Radiance compare H

  13. The variational method allows model radiances to be compared directly to observed radiancesEnables use of advanced observation operators Observation Satellite Radiance Model T and q H Model Radiance compare

  14. Jb: Ensures that the background model fields are adjusted meteorologically consistently in the region close to the observation location Increments due to a single observation of geopotential height at 1000hPa at 60N with value 10m below the background.

  15. Jb: The Balance Operator ensures the height and wind field balance is retained wind increments at 150 metre above surf. wind increments at 300hPa Increments for a single observation of geopotential height at 1000hPa. Left panel: wind increment near 300hPa. Right panel: wind increment about 150m above the surface.

  16. 4D-Var implementation used at ECMWF All observations within a 12-hour period (~9,000,000) are used simultaneously in one global (iterative) estimation problem • Observation minus model differences are computed at the observation time using the full forecast model at T1279 (16km) resolution • 4D-Var finds the 12-hour forecast evolution that optimally fits the available observations. A linearized forecast model is used in the minimization process based on the adjoint method • It does so by adjusting surface pressure, the upper-air fields of temperature, wind, specific humidity and ozone • The analysis vector consists of 80,000,000elements at T255 resolution (80km) 09Z 12Z 15Z 18Z 21Z

  17. In the incremental formulation the cost function J is expressed in terms of increments xwith respect to the background state, x=x-xb, at initial time. and are the TL of and , linearized around 4D-Var incremental formulationCourtier, Thépaut and Hollingsworth (1994) The innovations di are calculated using the non-linear operators, Hi and Mi . The i-summation is over N=25 ½-hour longtime slots of the 12-hour assimilation period. This ensures the highest possible accuracy for the calculation of the innovations di , which are the primary input to the assimilation!

  18. Implementation: Inner/outer iteration algorithm Hessian pre-conditioning Conjug. Gradient minimisation Improved TL approximations Multi-incremental T159/T255/T255 These developments facilitate: Use of higher density data Higher resolution inner loop (T255) and outer loop (T1279) Enhanced use of (relatively costly) TL physics Cloud and rain assimilation Revision of operational 4D-Var algorithm More work has been done to improve the representation of the smallest scales in the inner loop – i.e. the analysis increments.

  19. Multi-incremental quadratic 4D-Var at ECMWF T1279L91 T159L91 T255L91 T255L91

  20. 4D-Var with three outer loop:efficient, accurate and allows non-linearity • Analysis increments at 900hPa for temperature for each of the three minimizations. • Plot shows 2009 setup. From 2010: T95T159 and T799T1279. • Decreasing amplitude of increments: T95>T159>T255. • Last analysis step adds small corrections where data density is highest. • Note model and observation operators are re-linearized twice. T95 Add T95 increment to T799 background and re-linearize M, H T159 Add T159 increment and re-linearize M, H T255 Add T255 increment = final T799 analysis

  21. Physical processes in the ECMWF modelAn accurate model representation of the atmosphere is an important part of the assimilation system The atmosphere does not evolve in isolation, interactions between the atmosphere and the underlying land and ocean are also important in determining the weather. Ocean ice processes, ocean surface waves, land surface, soil, hydrological and snow processes are all represented at ECMWF in the most advanced operational Earth-system model available anywhere. These physical processes have smaller scales than the model grid (16 km) and are therefore represented by so-called “Parametrization Schemes” which represent the effect of the small-scale processes on the large-scale flow.

  22. Outer loop model resolution is now T1279L91 Important for accurate comparison against observations T799 (25km) 2006-2010 843,490 grid-points per level L91 L60 T1279 (16km) Since January 2010 2,140,704 grid-points per level

  23. Resolution and QC benefits analysis of extreme eventsImproved hurricane analyses due to Huber norm and resolution (35r3) Use of improved QC (Huber norm) 961hPa OPS (35r2) 980hPa Hurricane Bill, 20 Aug. 2009 Observed MSL pressure~944hPa (36r1) High-res system T1279+T159/T255/T255 945hPa

  24. Can we use our parallel computers efficiently? Scalability of IFS model – T1279 L91 Nodes Cores 1024 2048 3072 4096 5120 6144 7168 8192

  25. 4D-Var is more difficult to parallelize efficiently ideal T1279 model T1279 model T255an T1279obs T159an T1279obs =576 cores =1152 cores =1728 cores =2304 cores

  26. Parallel performance is important. Scalability is not perfect for the 4D-Var analysis 4D-Var T799 outer,T95/T255 inner, 91 levels on IBM system 512 768 1024 1536

  27. Hurricane Lili. Surface scatterometer winds. An example how 4D-Var propagates information vertically First guess MSL pressure Analysis MSL pressure MSL pressure Analysis increments NSCAT analysis No SCAT analysis S.M. Leidner, L. Isaksen and R.S. Hoffman ‘Impact of NSCAT Winds on Tropical Cyclones in the ECMWF 4DVAR assimilation system’ Mon. Wea. Rev. 131,1,3-26 (2003)

  28. Hurricane Lili. Surface scatterometer winds. An example how 4D-Var propagates information vertically First guess MSL pressure First guess MSL pressure Analysis MSL pressure MSL pressure Analysis increments NSCAT analysis No SCAT analysis S.M. Leidner, L. Isaksen and R.S. Hoffman ‘Impact of NSCAT Winds on Tropical Cyclones in the ECMWF 4DVAR assimilation system’ Mon. Wea. Rev. 131,1,3-26 (2003)

  29. Surface scatterometer wind information is propagated vertically and improve the analysis. Due to flow-dependent structure functions in 4D-Var

  30. 4D-Var is using more data from frequently reporting stations. The plots show the use of SYNOP surface pressure observations. Column height gives the number of observations available, while the black part displays those actually used in the assimilation. 4D-Var SYNOP Screening 4D-Var is using more a-synoptic data than 3D-Var 3D-Var SYNOP Screening 3D-Var is like 4D-Var without the time dimension. The analysis is performed at synoptic times only (0000, 0600, 1200 and 1800 UTC). Mostly only data valid a synoptic time is used.

  31. 4D-Var versus 3D-Var and Optimum Interpolation • 4D-Var is comparing observations with background model fields at the correct time • 4D-Var can use observations from frequently reporting stations • The dynamics and physics of the forecast model in an integral part of 4D-Var, so observations are used in a meteorologically more consistent way • 4D-Var combines observations at different times during the 4D-Var window in a way that reduces analysis error • 4D-Var propagates information horizontally and vertically in a meteorologically more consistent way • 4D-Var more complex: needs linearized perturbation forecast model and its adjoint to solve the cost function minimization problem efficiently

  32. Recent revisions to the assimilation system • Use of many new satellites and new instruments (will be presented by Peter Bauer tomorrow) • Variational bias correction of satellite radiances • Adaptive bias correction for radiosondes and SYNOP pressure data • Increased resolution from T799/T95/T159/T255 to T1279/T159/T255/T255 • More advanced Tangent Linear physics scheme in 4D-Var • Improved handling of supersaturation in the humidity analysis • Huber norm Variational Quality Control • Weak-constraint 4D-Var accounting for model error • Advanced diagnostic tools to understand impact of observations on analysis/forecast: Forecast Sensitivity to Observations

  33. Improved assimilation of satellite moisture data Assimilation of rain-affected microwave • First version (SSM/I radiances) 2005; extended to SSMIS, TMI, AMSR-E in 2007 • Direct 4D-Var radiance assimilation from March 2009 • Main difficulties: inaccurate moist physics parameterizations (location/intensity), formulation of observation errors, bias correction, linearity. 4D-Var first guess SSM/I Tb 19v-19h [K] SSM/I observational Tb 19v-19h [K]

  34. Humidity analysis improvements • New humidity background error model: • takes into account how humidity errors are affected by temperature errors in cloudy conditions (statistical relationship) • extension of the humidity errors to cater for supersaturated conditions (with respect to ice) Old New Changes in cloud cover spin-up. Monthly average zonal mean (Sep 2008) analysis minus a 15h forecast valid at the same time.

  35. Improved quality control of observationsHuber-norm observation cost function • L1 metric far from centre, L2 metric close to centre. • Robust method: a few erroneous observations does not ruin analysis • Adds some weight on observations with large departures • A set of observations with consistent large departures will influence the analysis • Most obs-bg innovations follow Huber-norm distribut. Gaussian Huber norm (New operational) Gaussian + flat (operational)

  36. How to account for model error in 4D-Var • First step is a weak-constraint 4D-Var that handles biases. Q only applied in stratosphere. Cost function including model error and bias: Model error η Model bias (cycled) ηb

  37. Weak-constraint 4D-Var formulation captures part of model error biases in the stratosphere AMSU-A first guess departures reduced

  38. Enhance diagnostics of the assimilation system • Forecast Sensitivity to Observations J is a measure ofthe forecast error (as defined through e.g. dry energy norm) Example: Impact of each observing system in ECMWF’s 2009 assimilation system

  39. Soil moisture assimilation using Ext. Kalman FilterWill be implemented in June 2010 Analysis increments (mm)

  40. Ensembles of data assimilationsIs being implemented now, first for EPS later for 4D-Var • Run an ensemble of analyses with randomly perturbed observations and SST fields, and form differences between pairs of analyses (and short-range forecast) fields. • These differences will have the statistical characteristics of analysis (and short-range forecast) error. To be used in specification of background errors = “errors of the day”. To indicate where good data should be trusted in the analysis (yellow shading). Also for initialization of the EPS!

  41. xb+εb xb+ηb xb+εb xb+ηb xb+ηb xb+εb Analysis Analysis Analysis Analysis Analysis Analysis Forecast Forecast Forecast Forecast Forecast Forecast Estimating Background Error Statistics from Ensembles of Data Assimilations (EDA) Run an ensemble of analyses with random observation and SST perturbations, plus stochastic model error representation. Form differences between pairs of background fields. These differences will have the statistical characteristics of background error (but twice the variance). Background differences

  42. Why implementing Ensemble Data Assimilation? • To estimate analysis uncertainty • To improve the initial perturbations in the Ensemble Prediction • To calculate static and seasonal background error statistics • To estimate flow-dependent background error in 4D-Var - “errors-of-the-day” • To improve QC decisions and improve the use of observations in 4D-Var

  43. Ongoingdevelopments in data assimilation at ECMWF • EKF for soil moisture analysis • EDA to provide flow-dependent background error information to 4D-Var • Vertical resolution increase planned for later in 2010 (~140+ levels TBD) • Long window 4D-Var: extend to 24 hour window, improve model error term • Improved assimilation of cloud/aerosols/rain observations • Flow-dependent data selection • Account for observation error correlations • Modularisation of the IFS and scalability of the assimilation system

  44. Summary of today’s lecture • Overview of the operational data assimilation cycle • Computational issues • Observations used by the ECMWF Assimilation System • Multi-incremental 4D-Var • Why is 4D-Var performing better than 3D-Var? • Recent improvements of ECMWF’s Assimilation System • Near future data assimilation implementations

More Related