Data Assimilation Strategies for Operational NWP at Meso-scale and Implication for Nowcasting

1. Data Assimilation Strategies for Operational NWP at Meso-scale and Implication for Nowcasting Thibaut Montmerle CNRM-GAME/GMAP WMO/WWRP Workshop on Use of NWP for Nowcasting UCAR, Boulder, CO, USA, 24-26 October, 2011

2. Introduction Non-hydrostatic models (in the 1-3 km horizontal resolution range) allow realistic representation of convection, clouds, precipitation, turbulence, surface interactions Such models have specific features that make their operational implementation difficult: • They need coupling models to provide LBC and surface conditions • They are expensive in computation time • Forecasts need to be frequently corrected towards observations to provide the best initial state possible through data assimilation (DA) Simulation of a MCS performed with AROME • The presence of many strongly non-linear processes, especially those related to diabatic phenomena, makes the DA task delicate

3. Introduction: requirements for Nowcasting Nowcasting algorithms require the best description of the atmospheric state at a particular time and the best very short term forecast available. => To answer to those specifications, operational NWP systems at convective scale have to: 1. provide the best analyses as possible frequently • The DA algorithm needs to be fast and efficient • A comprehensive set of observation types describing the clear air environment as well as convective systems should be used • The use of these observations must be as optimal as possible 2. provide physically meaningful forecasts • spin-up time must be as short as possible • maintain a realistic development of the analysed structures

4. Outlines 1. Introduction 2. Elements of NWP at convective scale - DA algorithms - error covariances - observations and observation operators 3. Example of NWP system at convective scale: AROME 4. Requirements for Nowcasting purposes 5. Conclusions

5. Elements of NWP at convective scale: DA algorithms DA aims in retrieving the best initial state (or analysis xa) from a previous forecast xb and from various observationsyo, the weight of these two entities being given by their respective error covariances B and R. If the model trajectory is supposed linear in the vicinity of the background, and if background and observation errors are decorrelated, the analysis is given by the BLUE equation: Where H is the non-linear observation operator that simulates the observation from the background, and H its linearized version. • B has a key role in DA, by smoothing and spreading the information brought by observations, and by propagating this information to other control variables through balance relationships. • Resolving this equation explicitely is infeasible because of the huge dimension of the system in meteorology • 2 different approaches can be considered to solve the BLUE: sequential or variational

6. At convective scale and for operational NWP, running an ensemble of forecasts in real time seems for the moment unaffordable Elements of NWP at convective scale: DA algorithms The sequential approaches These methods are based on the Kalman Filter that assimilates observations sequentially. Methods such as the EnKF (Evensen 1994) allow to compute B (or Pf ) in the BLUE by approximating the dispersion of an ensemble of forecast. Advantages: • easy to implement, well suited to parallel computing • avoid the computation of the TL/AD version of the model Drawbacks: • severe sampling noise in the raw covariances need to be empirically filtered which brings loss of information • sampling errors: the filter can collapse in case of misrepresentation of model error in the filter update • very expensive in computation time, especially for CRMs !

7. Albeit widely used at global scale,4DVar is very complex to implement at convective scale, especially because of the difficulty to linearize diabatic processes and other strongly non-linear parameterizations. Only JMA is running a 4DVar at 5 km resolution operationally Elements of NWP at convective scale: DA algorithms Variational approachessuch as 4DVar aim in seeking the minimum of a cost function, which satisfies theoritically the BLUE equation: Minimization is achieved using the adjoint method based on grad(J(x)) + All observations within a time window are used to find the new trajectory. + only one forecast/loop is performed -B is not flow-dependent but has to be calibrated beforehand - TL/AD version of the model has to be developed to compute grad(J(x))

8. 3DVar seems for the moment to be the best compromize for operational NWP at convective scale Elements of NWP at convective scale: DA algorithms At convective scale, most of operational NWP centers (MF, UKMO, NCEP…) use 3DVar schemes with short assimilation/forecast cycles to limit the gap in time between observations and the forecast to be corrected + Cheap, fast, no TL/AD of M - no integration in time: only observations valid around the analysis time are considered - as in 4DVar, B is not flow dependent

9. Elements of NWP at convective scale: DA algorithms Other possible aproaches: Hybrid EnVAR:modulation of the static B by filtered covariances computed from an ensemble 3DVarFGAT : allows to compare the observations and the background at the right time (as in 4DVar), but supposes that innovations (obs-guess) are constant in time (as 3DVar: no TL/AD of M) => Suitable for moving observations, not for static measurements for which the analysis is like a “mean” of the successive innovations Nudging Latent Heat Nudging (LHN) allows to retrieve atmospheric profiles consistent with radar derived precipitation rates (oper at UKMO and DWD). RR uses a cloud analysis in order to update cloudy variables that are not considered in the 3DVar. Although retrieved fields are subject to possible inconsistencies with analyzed ones from VAR, nudging is a simple way to take observations of hydrometeors into account.

10. Elements of NWP at convective scale: error covariances Representation of the background error covariances matrix B: • The “true” state needed to measure error against is unknown. Differences between forecasts (from different forecast terms or from an ensemble) are generally considered to mimic climatological forecast errors. • Because of its size, B can be neither estimated at full rank nor stored explicitly => covariances have to be modelled B is generally splitted in one balance operator (e.g. geostrophic balance) and one spatial transform, aiming in projecting each parameter onto uncorrelated spatial modes, and then in dividing by the square root of the variance of each mode

11. B « rainy » Elements of NWP at convective scale: error covariances Main challenges at convective scale for B: • Balance constraints, that were initially developed for DA in global models, are likely to be inadequate, especially in convection B OPER Montmerle and Berre (2010) have for instance shown that forecast errors in rain strongly differ from climatological values and reflect diabatic processes The gap with the geostrophic balance in B increases with precipitation intensities (Carron and Fillon (2010)) • Forecast errors of variables linked to clouds and precipitations are inhomogeneous, anisotropic and flow dependent => Works, mainly based on ensembles, are ongoing in order to add flow dependency to B Cross covariances between forecast errors of q (y-axis) and divu (x-axis)

12. Elements of NWP at convective scale: error covariances Much less attention is given to the representation of the observation error covariances matrix R: • diagonal elements (variances) must represent instrumental errors, representativeness and precision errors of H • off-diagonal (spatial or inter-channel correlations) are generally neglected. They can however be inferred by a comparison with other observations or with the background (innovations). To ensure the basic hypotheses of the BLUE, a spatial thinning and/or an inflation of variances are applied to prevent possible observation error correlations between adjacent pixels. It has however been shown that correlated observations are less informative than uncorrelated observations, even if R is well specified IASI inter-channel correlations (Bormann et al, ECMWF, 2011)

13. At convective scale, observations performed in clouds and precipitations such as cloudy radiances and Doppler radar data are of great interest because of the explicit representation of convection. Elements of NWP at convective scale: observation operators To consider an observation type in DA, the observation operator H and its TL/AD versions have to be developed. Example of the main observation types considered in the operational AROME-France model

14. Shifting of the main convergence line Elements of NWP at convective scale: observation operators Example: radial velocities OBS: Z & Wind retrieval With Vr Without Vr AROME Low-level divergence analyses Bousquet, Montmerle and Tabary 2007

15. 1D+3DVar (Caumont et al. 2010): Allows to consider volumic observations. Operational at MF since spring 2010 and recently at JMA Elements of NWP at convective scale: observation operators Radar reflectivities: Very difficult task: weak correlation with hydrometeor contents, strongly nonlinear microphysical processes to be included in H , non-Gaussian error distributions, complex forecast errors. • Methods that use precipitation rates: Diabatic DFI and LHN Other methods allowing to retrieve hydrometeor contents using a nudging approach or directly in 3DVar (Xiao et al (2007)) or in 4DVar (Sun and Crook (1997)) have also been tested on case studies.

16. Elements of NWP at convective scale: observation operators Illustration of the 1D+3DVar assimilation of radar reflectivities (E. Wattrelot) Specific humidity increment With Z Without Z 6h UTC Radar Simulated Z - 3h forecast range 9h UTC

17. At convective scale: • Geostationary satellites benefit from their high temporal resolution but suffer from their weak spectral resolution compared to radiometers such as IASI and AIRS • positive impact has been found while assimilating data from polar orbiting satellites, the information brought through DA being propagated by the cycling => Satellite radiances are of great interest to describe pre-convective environmental conditions Elements of NWP at convective scale: observation operators Satellite observations • are the main observation type in terms of number and impact in global models • IR and MW radiances allow to retrieve q and T profiles with a vertical resolution that depends on the spectral resolution of the instrument • Radiances are strongly sensitive to surface conditions and IR measurements are contaminated by clouds • other very interesting products such as winds derived from scatterometers or AMV, integrated q from GPSRO, cloud types... Guidard et al., 2011

18. Outlines 1. Introduction 2. Elements of NWP at convective scale - DA algorithms - error covariances - observations and observation operators 3. Example of NWP system at convective scale: AROME 4. Requirements for Nowcasting purposes 5. Conclusions

19. Example of operational NWP system : AROME-France • The non-hydrostatic AROME NWP system became operational at the end of 2008 with a 2.5 km horizontal resolution • Lateral boundaries are provided by the global ARPEGE model that has a horizontal resolution around 10 km over France AROME France domain ARPEGE Global domain

20. ARPEGE (short cut-off) 102h 72h 84h 60h LBCs 06 18 00 UTC 12 30h 30h 30h 30h AROME 3h Guess 3h 3h 3h Guess 06 18 00 UTC 12 09 03 15 21 [ ] [ ] [ ] [ ] [ ] [ ] [ ] +/- 1h30 Obs validity time => Observations with high temporal frequencies (radars, surface, geostationnary radiances…) are clearly under-exploited for the moment Example of operational NWP system : AROME-France • 3h assimilation/forecast cycles, own surface analysis • No temporal dimension in 3DVar: only the closest observations to the assimilation time are considered A temporal gap between forecast and observations from moving platforms is allowed within the +/- 1h30 interval

21. RADAR Z RADAR Vr Sat IASI Aircraft SEVIRI TEMP Surface Percentage of observations used in AROME– August 2011 Example of operational NWP system : AROME-France • Used observations : 24 Doppler radars, growing number of satellite data Time evolution of monthly values since sept. 2008 Efforts are ongoing to assimilate more GPS ZTD data, more channels over land (smaller thinning boxes, better characterization of surface conditions) and to consider cloudy radiances in DA.

22. Example of operational NWP system : AROME-France • Example of forecast : 6th of October 2011 r12, 12h simulation. Narrow frontal rainband evolving in stratiform precipitations AROME Z850hPa Radar Mosaic

23. Example of operational NWP system : AROME-France • Example of forecast : 26th of Aug. 2011 r12, 12h simulation Complex LS circulation: Frontal rainband associated with a cyclogenesis over France, stationary convection due to orography in the Rhône basin => Such forecasts clearly demontrates the potential of convective scale NWP for nowcasting applications

24. Outlines 1. Introduction 2. Elements of NWP at convective scale - DA algorithms - error covariances - observations and observation operators 3. Example of NWP system at convective scale: AROME 4. Requirements for Nowcasting purposes 5. Conclusions

25. Requierements for Nowcasting purposes Current NWP systems at convective scales are not optimal for Nowcasting applications, especially because of too late availability of analyses and forecasts due to : • the waiting of LBCs from the coupling model => forecasts from former analysis time must be used • the cut-off time => some observations must be “sacrified” and/or shorter cycle frequencies must be considered • the computational time of analysis and forecast => smaller domains or “older” forecast must be used A degraded version of AROME has been developed with these specifications (AROME-PI: remember Ludovic Auger’s talk) • The spin-up of the model => Initialization procedures and more optimal background error covariances must be applied

26. (dPs/dt)OPER – (dPs/dt)EXP) vs. % of rainy pts in mask Corel=0.64 Requierements for Nowcasting purposes Spin-up reduction 1/2 Lapse of time necessary for the model’s equation to be in balance. Can be due to : • Inconsistency between LBCs and the model’s state inside the computational domain => Can be reduced by considering the analysis as coupling file at t=0 • imbalances of analysis increments =>Improvements can be found using ensemble assimilation to compute B => Flow-dependency of the balance operator should be better represented in B Here OPER uses a climatological B matrix, whereas EXP uses in addition forecast errors representative of precipitations exclusively in rain (following the heterogeneous B matrix formulation of Montmerle and Berre (2010)) OPER EXP Noise = mean absolute sfc pressure tendency (hPa/h)

27. Requierements for Nowcasting purposes Spin-up reduction 2/2 • imbalances between analyzed and non-analyzed fields (e.g. dynamics, T and q vs. microphysical or NH variables) => The resulting numerical waves can be filtered out by digital filters (DFI or incDFI) or by adding fractions of increments within the assimilation window (Incremental Analysis Update, operational at UKMO) P. Brousseau Efficient methods to reduce spin-up but, for AROME, forecast scores are degraded so far. Small scale analyzed structures (e.g low level convergence) are also considerably smoothed.

28. Requierements for Nowcasting purposes 1h-cycle • Allows to consider observations with high temporal frequencies while keeping a 3DVar assimilation system • possible if spin-up is sufficiently reduced Operational in RUC/RR. At MF, tests are ongoing using IAU or DFI but so far, scores are degraded compared to 3h-cycle. Possible explanations: • cycling of residual numerical waves • difficulty to tune values of observation vs. background errors obs-analysis obs-guess 1h cycle 3h cycle P. Brousseau

29. Outlines 1. Introduction 2. Elements of NWP at convective scale - DA algorithms - error covariances - observations and observation operators 3. Example of NWP system at convective scale: AROME 4. Requirements for Nowcasting purposes 5. Conclusions

30. Conclusions Feedback from operational NWP at convective scale: • 3DVar used at high frequency is the most frequent choice for DA because of its low computational cost and its (relative) simplicity • Adequate balances and flow dependency is needed in B to optimize the use of observations and to reduce spin-up • Efforts still are needed to consider observations linked to cloud and precipitations (especially cloudy radiances and radars) in a more optimal way in DA For the time being, only degraded version of such systems can be used in Nowcasting algorithms, mainly for a question of forecast availability. These versions use shorter cut-off time, shorter cycle frequencies, asynchronous coupling files, and eventually smaller computational domains. In the infra-hour range, spin-up can be problematic and forecasted microphysical quantities should be considered with caution. However, methods to reduce this spin-up exist but often tend to degrade forecast scores

31. Thank you for your attention!

32. References Brousseau, P.; Berre, L.; Bouttier, F. & Desroziers, G. : 2011. Background-error covariances for a convective-scale data-assimilation system: AROME France 3D-Var. QJRMS., 137, 409-422 Berre, L., 2000: Estimation of synoptic and mesoscale forecast error cavariances in a limited area model. MWR. 128, 644–667. J.-F Caron and L. Fillion. An examination of background error correlations between mass and rotational wind over precipitation regions. Mon. Wea. Rev., 138 :563–578, 2010. O. Caumont, V. Ducrocq, E. Wattrelot, G. Jaubert, and S. Pradier-Vabre. 1d+3dvar assimilation of radar reflectivity data : a proof of concept. Tellus, 62, 2010. V. Guidard, N. Fourrié, P. Brousseau, and F. Rabier: 2011. Impact of iasi assimilation at global and convective scales and challenges for the assimilation of cloudy scenes. in press. Quart. J. Roy. Meteor. Soc. Montmerle T, Berre L. 2010. Diagnosis and formulation of heterogeneous background-error covariances at themesoscale. QJRMS, 136, 1408–1420. Xiao, Q et al, 2007: An Approach of Radar Reflectivity Data Assimilation and Its Assessment with the Inland QPF of Typhoon Rusa (2002) at Landfall. JAMC, 46, 14-22.