Parametrizations and data assimilation

1. Parametrizations and data assimilation Marta JANISKOVÁ ECMWF marta.janiskova@ecmwf.int

2. Parametrization = description of physical processes in the model. • Why is physics needed in data assimilation ? • How the physics is applied in variational data assimilation system ? • Which parametrization schemes are used at ECMWF ? • What are the problems to be solved before using physics in data • assimilation ? • What is an impact of including the physical processes in assimilating model ? • How the physics is used for assimilation of observations related to the physical processes ?

3. the better the assimilating model • (4D-Var consistently using the information coming from the observations and the model) the better the analysis (and the subsequent forecast) • the more sophisticated the model • (4D-Var containing physical parametrizations) the more difficult the minimization (on-off processes, non-linearities) DEVELOPMENT OF A PHYSICAL PACKAGE FOR DATA ASSIMILATION = FINDING A TRADE-OFF BETWEEN: Simplicity and linearity (using simplified linear model is a basis of incremental variational approach) Realism POSITION OF THE PROBLEM IMPORTANCE OF THE ASSIMILATING MODEL

4. IMPORTANCE OF INCLUDING PHYSICSIN THE ASSIMILATING MODEL • using an adiabatic linear model can be critical especially in the tropics, • planetary boundary layer, stratosphere • missing physical processes in assimilation systems can lead to the so-called • spin-up/spin-down problem • using the adjoint of various physical processes should provide an initial • atmospheric state for NWP models which is: • more consistent with physical processes • producing better agreement between the model and data • inclusion of such processes is a necessary step towards: • initialization of prognostic variables related to physical processes • the use of new (satellite) observations in data assimilation systems • (rain, clouds, soil moisture, …)

5. STANDARD FORMULATION OF 4D-VAR • the goal of 4D-Var is to define the atmospheric statex(t0) such that the “distance” between • the model trajectory and observations is minimum over a given time period [t0, tn] finding the model state at the initial time t0 which minimizes a cost-function J : H is the observation operator (model space  observation space) xi is the model state at time step ti such as: M is the nonlinear forecast model integrated between t0 and ti

6. In incremental 4D-Var, the cost function is minimized in terms of increments: Tangent-linear operators  tangent linear model with the model state defined at any timetias: • 4D-Var can be then approximated to the first order as minimizing: where is theinnovation vector Adjoint operators • Gradient of the cost function to be minimized:  computed with the non-linear model at high resolution using full physics  M  computed with the tangent-linear model at low resolution using simplified physics  M’  computed with a low resolution adjoint model using simplified physics  MT INCREMENTAL FORMULATION OF 4D-VAR

7. VALIDATION OF THE LINEARIZED SCHEMES • classical one(TL - Taylor formula, AD - test of adjoint identity) • examination of the accuracy of the linearization - TL integrations propagating in time finite-size analysis increments with the trajectory computed using the full nonlinear model Comparison: finite differences  tangent-linear integration

8. deep convection large-scale condensation to be replaced by to be replaced by mass flux convection statistical cloud scheme already used in 1D-Var precipitation assimilation moist processes ECMWF LINEARIZED PHYSICS vertical diffusion gravity wave drag radiation dry processes

9. SIMPLIFICATIONS OF THE LINEARIZED PHYSICS • done with the aim to have a physical package for data assimilation: • simple – for the linearization of the model equations • regular – to avoid strong non-linearities and thresholds • effective – to reduce the computational cost for operational applications

10. IMPORTANCE OF THE REGULARIZATION OF TL MODEL • physical processes are characterized by: • threshold processes: • discontinuities of some functions describing the physical processes (some on/off processes: condensation, precipitation formation, …) • discontinuities of the derivative of a continuous function • strong nonlinearities

11. Without adequate treatment of most serious threshold processes, the TL approximation can turn to be useless. WHY REGULARIZATION IS IMPORTANT

12. dyNL dyTL dx Potential source of problem (example of precipitation formation)

13. dyTL2 Possible solution, but … dyTL1 dyNL dx

14. dyTL3 dx2 … may just postpone the problem and influence the performance of NL scheme dyTL2 dyNL dx1

15. dyTL2 However, the better the model  the smaller the increments dyTL1 dyNL dx

16. In NWP – a tendency to develop more and more sophisticated physical parametrizations they may contain more discontinuities For the “perturbation” model – more important to describe basic physical tendencies while avoiding the problem of discontinuities Level of simplifications and/or required complexity depends on: • which level of improvement is expected (for different variables, vertical and horizontal resolution, …) • which type of observations should be assimilated • necessity to remove threshold processes Different ways of simplifications: • development of simplified physics from simple parametrizations used in the past • selecting only certain important parts of the code to be linearized EXAMPLES OF REGULARIZATIONS AND SIMPLIFICATIONS (1)

17. Function of the Richardson number • 0.010 • 0.000 f(Ri) • -0.010 • -0.020 • reduced perturbation of the exchange coefficients(Janisková et al., 1999): • original computation of Ri modified in order to modify/reduce f’(Ri), or • reducing a derivative, f’(Ri), by factor 10 in the central part (around the point of singularity ) -20. -10. 0. 10. 20. Ri number EXAMPLES OF REGULARIZATIONS AND SIMPLIFICATIONS (2) Regularization of vertical diffusion scheme: • perturbation of the exchange coefficients(which are function of the Richardson number Ri)is neglected, K’ = 0(Mahfouf, 1999)

18. EXAMPLES OF REGULARIZATIONS AND SIMPLIFICATIONS (3) • reduction of the time step to 10 seconds to guarantee stable time integrations • of the associated TL model(Zhu and Kamachi, 2000) • selective regularization of the exchange coefficients K based on the linearization • error and a criterion for the numerical stability(Laroche et al., 2002)

19. EXAMPLES OF REGULARIZATIONS AND SIMPLIFICATIONS (4) operational version of ECMWF SW: 2 spectral intervals LW:neural network+mean Jacob.

20. physical processes are characterized by: • threshold processes: • discontinuities of some functions describing the physical processes (some on/off processes) • discontinuities of the derivative of a continuous function • strong nonlinearities • regularizations help to remove the most important threshold processes in physical parametrizations which can effect the range of validity of the tangent linear approximation • after solving the threshold problems clear advantage of the diabatic TL evolution of errors compared to the adiabatic evolution IMPORTANCE OF THE REGULARIZATION OF TL MODEL

21. TANGENT-LINEAR DIAGNOSTICS Comparison: finite differences (FD)  tangent-linear (TL) integration

22. Zonal wind increments at model level ~ 1000 hPa[ 24-hour integration] FD TLADIAB TLADIABSVD TLWSPHYS • TLADIAB – adiabatic TL model • TLADIABSVD – TL model with very simple vertical diffusion (Buizza 1994) TLWSPHYS – TL model with the whole set of simplified physics (Mahfouf 1999)

23. Comparison: finite differences (FD)  tangent-linear (TL) integration Diagnostics: • mean absolute errors: • relative error TANGENT-LINEAR DIAGNOSTICS

24. Temperature Impact of very simple vertical diffusion scheme EXP - REF 10 20 30 40 50 60 REF = ADIAB 80N 60N 40N 20N 0 20S 40S 60S 80S relative improvement [%] EXP adiabsvd

25. Temperature Impact of dry physical processes EXP - REF 10 20 30 40 50 60 REF = ADIAB 80N 60N 40N 20N 0 20S 40S 60S 80S relative improvement [%] EXP adiabsvd || vdif + gwd + radold

26. Temperature Impact of dry + moist physical processes EXP - REF 10 20 30 40 50 60 REF = ADIAB 80N 60N 40N 20N 0 20S 40S 60S 80S relative improvement [%] EXP adiabsvd || vdif + gwd + radold + lsp + conv

27. Temperature Impact of physical processes (including improved schemes) EXP - REF 10 20 30 40 50 60 REF = ADIAB 80N 60N 40N 20N 0 20S 40S 60S 80S relative improvement [%] X X EXP adiabsvd || vdif + gwd + conv + radnew + cl_new

28. Impact of all physical processes EXP - REF Zonal wind [%] relative improvement X X EXP - REF Specific humidity relative improvement [%] X X

29. FORECAST VERIFICATION – 500 hPa GEOPOTENTIAL period: 15/11/2000 – 13/12/2000 root mean square error – 29 cases N.HEM S.HEM oper RD 4v no phys IMPACT OF THE LINEARIZEDPHYSICSIN 4D-VAR (1) • comparisons of the operational version of 4D-Var against the version without linearized physics included shows: • positive impact on analysis and forecast

30. IMPACT OF THE LINEARIZEDPHYSICSIN 4D-VAR (2) • reducing precipitation spin-up problem when using simplified physics in 4D-Var minimization Time evolution of total precipitation in the tropical belt [30S, 30N] averaged over 14 forecasts issued from 4D-Var assimilation

31. 75.4 63.8 -30.3 -23.3 -17.9 -10.0 IMPACT OF THE LINEARIZEDPHYSICSIN 4D-VAR (3) 1-DAY FORECAST ERROR OF 500 hPa GEOPOTENTIAL HEIGHT OPER (very simple radiation) vs. NEWRAD (new linearized radiation) (27/08/2001 t+24) A2: FC_NEWRAD – ANAL_OPER A1: FC_OPER – ANAL_OPER A2 – A1 impact of new linearized radiation

32. Observation term Background term • The minimization requires an estimation of the gradient of the cost function: • The operator HTcan be obtained: • explicitly (Jacobian matrix) • using the adjoint technique 1D-Var assimilation of observations related to the physical processes • For a given observation yo, 1D-Var searches for the model state x=(T,qv) that minimizes the cost function: B = background error covariance matrix R = observation and representativeness error covariance matrix H = nonlinear observation operator (model space  observation space) (physical parametrization schemes, microwave radiative transfer model, reflectivity model, …)

33. physics output physical parametrization schemes control variables increments adjoint of physical parametrization schemes Schematic description of forward modelling of physical outputs and their corresponding adjoint modelling control variables T, q, ps physics output departures

34. More recent developments: • New simplified convection scheme (Lopez & Moreau 2005) • New simplified cloud scheme (Tompkins & Janisková 2004) used in 1D-Var • Microwave Radiative Transfer Model (Bauer, Moreau 2002) • Assimilation experiments of direct measurements from TRMM and SSM/I (TB or Z) instead of indirect retrievals of rainfall rates, in a ‘1D-Var + 4D-Var’ framework. OPERATIONAL USE: assimilation of precipitation affected microwave radiancess since June 2005 (Bauer et al. 2006) Precipitation assimilation at ECMWF Goal:To assimilate observations related to precipitation and clouds in ECMWF 4D-Var system including parameterizations of atmospheric moist processes. A bit of history: • Work on precipitation assimilation at ECMWF initiated by Mahfouf and Marécal. • 1D-Var on TMI and SSM/I rainfall rates (RR) (M&M 2000). • Indirect ‘1D-Var + 4D-Var’ assimilation of RR more robust than direct 4D-Var. • ‘1D-Var + 4D-Var’ assimilation of RR is able to improve humidity but also the dynamics in the forecasts (M&M 2002). TMI – TRMM Microwave Imager, TRMM – Tropical Rainfall Measuring Mission SSM/I – Special Sensor Microwave/Imager

35. TMI TBs or TRMM-PR reflectivities Retrieval algorithm (2A12,2A25) “Observed” rainfall rates Observations interpolated on model’s T511 Gaussian grid moist physics + radiative transfer moist physics 1D-Var “TCWVobs”=TCWVbg+∫zqv background T,qv background T,qv 4D-Var “1D-Var+4D-Var” assimilation of observations related to precipitation 1D-Var on TBs or reflectivities 1D-Var on TMI or PR rain rates

36. Background 1D-Var/TB 1D-Var/RR PATER PATER obs 1D-Var on TMI data(Lopez and Moreau, 2003) Tropical Cyclone Zoe (26 December 2002 @1200 UTC) 1D-Var on TMI Rain Rates / Brightness Temperatures Surface rainfall rates (mm h-1)

37. 2A25 Rain Background Rain 1D-Var Analysed Rain 2A25 Reflect. Background Reflect. 1D-Var Analysed Reflect. 1D-Var on TRMM/ Precipitation Radar data(Benedetti and Lopez, 2003) Tropical Cyclone Zoe (26 December 2002 @1200 UTC) Vertical cross-section of rain rates (top, mm h-1) and reflectivities (bottom, dBZ): observed (left), background (middle), and analysed (right). Black isolines on right panels = 1D-Var specific humidity increments.

38. 1D+4D-Var assimilation of microwave radiances affected by precipitation (Bauer et al. 2006) rms error differences between RAIN(run with precipitation assimilation) and REF(reference run) geopotential height at 850 hPa (t+24) – mean over 30 days (September 2004) negative values = RAIN improved w.r.t. REF

39. | FG - OBS | - | ANAL - OBS | TCWV LWD SWD LWP positive values = improvement 1D-Var assimilation of cloud related ARM observations(1)(Janisková et al., 2002) ARM SGP, May 1999 - observations: - surface downward longwave radiation (LWD), - total column water vapour (TCWV) - cloud liquid water path (LWP) Observation operator includes: - shortwave and longwave radiation schemes - diagnostic cloud scheme

40. 1D-Var assimilation of cloud related ARM observations (2)(Benedetti and Janisková, 2004) • Cloud reflectivity in dBZ/10 • (retrieved from 35 GHz radar) • period: January 2001 > 20 dBZ 0 dBZ<Z< 10 dBZ -10 dBZ<Z< 0 dBZ < -20 dBZ OBS FG AN ARM – Atmospheric Radiation Measurement

41. 1D-Var of cloud related ARM observations (3a)(Janisková 2004, Benedetti & Janisková 2004) Synergy of active and passive observations for cloud assimilation AN1= Reflectivity only AN2= Total Column Water Vapor + Surface LW radiative flux AN3 = Reflectivity + Total Column Water Vapor + Surface LW radiative flux RELATIVE HUMIDITY BIAS The combination of passive and active observations gives the lowest analysis bias at all levels with respect to radiosoundings %

42. 2D-Var x0 = profile of temperature and specific humidity at initial time B = error covariance matrix for background Ri = error covariance matrices for observations at time ti Hi = “observation operator” (model space  observation space) = OMti i 2D-Var assimilation of ARM observations affected by clouds and precipitation (Lopez et al., 2005) • Main objective: to investigate the possibility to assimilate observations affected by cloud and/or precipitation over a certain time window (12 hours), using measurements either with a high temporal resolution (30 min) or that are accumulated or averaged in time • 2D-Var framework using ECMWF Single Column Model (SCM, M. Köhler) → much more simple to run and to interpret than 4D-Var • So far, test with MW TBs, cloud radar reflectivities, rain-gauge and GPS TCWV data

43. Conclusions: – Assimilation of half-hourly MW TBs over 12 hours is feasible, provided a proper screening is applied ( i.e. |guess – obs| < 3 K ) – Better to assimilate reflectivity profiles averaged over the assimilation window (despite the obvious lost of information) than half-hourly reflectivity profiles due to the sometimes large model-obs departures at such “high” temporal resolution 2D-Var on 23.8 and 31.4 GHz TBs & mean profiles of cloud radar reflectivities (Lopez et al., 2005) TCWV 23.8 GHz TB ARM SGP site 4 February 2001 31.4 GHz TB Mean Reflectivity

44. SUMMARY AND PERSPECTIVES • Positive impact from including linearized physical parametrization schemes into the assimilating model has been demonstrated. • Physical parametrizations become important components in current variational data assimilation systems • Recently, improvement/development of the schemes for the linearized moist processes with emphasis on: • moist convection (Lopez and Moreau 2005) • parametrization of large-scale condensation and clouds (Tompkins and Janisková 2004) • Some care must be taken when deriving the linearized parametrization schemes linearity, regularity & efficiency  realism • This is particularly true for the assimilation of observations related to precipitation, clouds and soil moisture, to which a lot of effort is currently devoted.

45. REFERENCES Bauer, P., Lopez, P., Benedetti, A., Salmond, D. and Moreau, E., 2006: Implementation of 1D+4D-Var assimilation of precipitation affected microwave radiances at ECMWF. Part I: 1D-Va. Quart. J. Roy. Meteor. Soc., accepted. Bauer, P., Lopez, P., Salmond, D., Benedetti, A., Saarinen, S. and Bonazzola, M., 2006: Implementation of 1D+4D-Var assimilation of precipitation affected microwave radiances at ECMWF. Part II: 4D-Var. ECMWF Technical Memorandum 488 Benedetti, A. and Janisková, M., 2004: Advances in cloud assimilation at ECMWF using ARM radar data. Extended abstract for ICCP, Bologna 2004 Errico, R.M., 1997: What is an adjoint model. Bulletin of American Met. Soc., 78, 2577-2591 Fillion, L. and Errico, R., 1997: Variational assimilation of precipitation data using moist convective parametrization schemes: A 1D-Var study. Mon. Wea. Rev., 125, 2917-2942 Fillion, L. and Mahfouf, J.-F., 2000: Coupling of moist-convective and stratiform precipitation processes for variational data assimilation. Mon. Wea. Rev., 128, 109-124 Klinker, E., Rabier, F., Kelly, G. and Mahfouf, J.-F., 2000: The ECMWF operational implementation of four-dimensional variational assimilation. Part III: Experimental results and diagnostics with operational configuration. Quart. J. Roy. Meteor. Soc., 126, 1191-1215 Chevallier, F., Bauer, P., Mahfouf, J.-F. and Morcrette, J.-J., 2002: Variational retrieval of cloud cover and cloud condensate from ATOVS data. Quart. J. Roy. Meteor. Soc., 128, 2511-2526 Chevallier, F., Lopez, P., Tompkins, A.M., Janisková, M. and Moreau, E., 2004: The capability of 4D-Var systems to assimilate cloud_affected satellite infrared radiances. Quart. J. Roy. Meteor. Soc., in press Janisková, M., Thépaut, J.-N. and Geleyn, J.-F., 1999: Simplified and regular physical parametrizations for incremental four-dimensional variational assimilation. Mon. Wea. Rev., 127, 26-45 Janisková, M., Mahfouf, J.-F., Morcrette, J.-J. and Chevallier, F., 2002: Linearized radiation and cloud schemes in the ECMWF model: Development and evaluation. Quart. J. Roy. Meteor. Soc.,128, 1505-1527

46. REFERENCES Janisková, M., Mahfouf, J.-F. and Morcrette, J.-J., 2002: Preliminary studies on the variational assimilation of cloud-radiation observations. Quart. J. Roy. Meteor. Soc., 128, 2713-2736 Janisková, M., 2004: Impact of EarthCARE products on Numerical Weather Prediction. ESA Contract Report, 59 pp. Laroche, S., Tanguay, M. and Delage, Y., 2002: Linearization of a simplified planetary boundary layer parametrization. Mon. Wea. Rev., 130, 2074-2087 Lopez, P. and Moreau, E., 2005: A convection scheme for data assimilation purposes: Description and initial test. Quart. J. Roy.. Meteor. Soc., 131, 409-436 Lopez., P., Benedetti, A., Bauer, P., Janisková, M. and Köhler, M.., 2005: Experimental 2D-Var assimilation of ARM cloud and precipitation observations. ECMWF Technical Memorandum 456 Mahfouf, J.-F., 1999: Influence of physical processes on the tangent-linear approximation. Tellus, 51A, 147-166 Marécal, V. and Mahfouf, J.-F., 2000: Variational retrieval of temperature and humidity profiles from TRMM precipitation data. Mon. Wea. Rev., 128, 3853-3866 Marécal, V. and Mahfouf, J.-F., 2002: Four-dimensional variational assimilation of total column water vapour in rainy areas. Mon. Wea. Rev., 130, 43-58 Marécal, V. and Mahfouf, J.-F., 2003: Experiments on 4D-Var assimilation of rainfall data using an incremental formulation. Quart. J. Roy. Meteor. Soc., 129, 3137-3160 Moreau, E., Lopez, P., Bauer, P., Tompkins, A.M., Janisková, M. And Chevallier, F., 2004: Rainfall versus microwave brightness temperature assimilation: A comparison of 1D-Var results using TMI and SSM/I observations. Quart. J. Roy. Meteor. Soc., in press. Tompkins, A.M. and Janisková, M. , 2004: A cloud scheme for data assimilation: Description and initial tests. Quart. J. Roy. Meteor. Soc., 130, 2495-2518 Zhu, J. and Kamachi, M., 2000: The role of the time step size in numerical stability of tangent linear models. Mon. Wea. Rev., 128, 1562-1572