1 / 34

Filtering of variances and correlations by local spatial averaging

Filtering of variances and correlations by local spatial averaging. Loïk Berre Météo-France. Outline. 1. Contrast between two extreme approaches in Var/EnKF ? 2.The spatial structure of sampling noise and signal 3. Spatial averaging of ensemble-based variances

sloan
Download Presentation

Filtering of variances and correlations by local spatial averaging

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Filtering of variances and correlationsby local spatial averaging Loïk Berre Météo-France

  2. Outline 1. Contrast between two extreme approaches in Var/EnKF ? 2.The spatial structure of sampling noise and signal 3. Spatial averaging of ensemble-based variances 4. Spatial averaging of innovation-basedvariances 5. Spatial averaging of correlations

  3. Two usual extreme approaches in Var/EnKF Covariance modelling : • Var: often globally averaged (spatially). robust with a very small ensemble, but lacks heterogeneity completely. • EnKF: often « purely local ».  a lot of geographical variations potentially, but requires a rather large ensemble, and it ignores spatial coherences. An attractive compromise is to calculate local spatial averages of covariances.

  4. Is the usual ensemble covariance optimal ? • Usual estimation (« raw »): B(N) = 1/(N-1) Si eb(i) eb(i)T = best estimate, for a given ensemble size N ? • No. Better to account for spatial structures of sampling noise and signal. Similar to accounting for spatial structures of errors in data assimilation, through B and R.

  5. The spatial structure ofsampling noise & signalin ensemble variance fields

  6. Spatial structure of sampling noise(Fisher and Courtier 1995 Fig 6, Raynaud et al 2008a) True variance field V* ~ large scale  Sampling noise Ve=V(N)-V* ~ large scale ? NO. N = 50 L( eb ) = 200 km  While the signal of interest is large scale, the sampling noise is rather small scale.

  7. Spatial structure of sampling noise(Raynaud et al 2008b) Spatial covariance of sampling noise Ve=V(N)-V* : Ve (Ve)T = 2/(N-1) B* ° B* where B* ° B* is the Hadamard auto-product of B* = eb (eb )T . • The spatial structure of sampling noise Ve is closely connected to the spatial structure of background errorseb.

  8. Spatial structure of sampling noise(Raynaud et al 2008b) Correlation function of sampling noise: cor(Ve[i], Ve[j]) = cor(eb[i], eb[j])² Length-scale L( Ve ) of sampling noise: L( Ve ) = L( eb ) / 2 • The sampling noise Ve is smaller scale than the background error field eb… … which is smaller scale than the variance field V* of background error?

  9. Spatial structure of signal(Houtekamer and Mitchell 2003, Isaksen et al 2007) Large scale features (data density contrasts, synoptic situation, …) tend to predominate in ensemble-based sigmab maps, which indicates that the signal of interest is large scale.

  10. Spatial structure of sampling noise & signal(Isaksen et al 2007) • General expectation : increasing the ensemble size reduces sampling noise, whereas the signal remains. N = 10 N = 50 • Experimental result : when increasing the ensemble size, small scale details tend to vanish, whereas the large scale part remains. • This indicates/confirms that the sampling noise is small scale, and that the signal of interest is large scale.

  11. COMPARISON BETWEEN TWO “RAW” sb MAPS (Vor, 500 hPa) FROM TWO INDEPENDENT 3-MEMBER ENSEMBLES « RAW » sb ENS #1 « RAW » sb ENS #2 Differences correspond to sampling noise, which appears to be small scale. Common features correspond to the signal, which appears to be large scale.

  12. Optimized spatial averaging of ensemble-based variances

  13. “OPTIMAL” FILTERING OF THE BACKGROUND ERROR VARIANCE FIELD Accounting for spatial structures of signal and noise leads to the application of a low-pass filter r (as K in data assim°): r Vb* ~ r Vb with r=signal / (signal+noise) (Raynaud et al 2008b)

  14. “OPTIMAL” FILTERING OF THE BACKGROUND ERROR VARIANCE FIELD SIMULATED « TRUTH » FILTERED SIGMAB’s (N = 6) (Raynaud et al 2008a) RAW SIGMAB’s (N = 6)

  15. RESULTS OF THE FILTERING (REAL ENSEMBLE ASSIMILATION) sb ENS 1 « FILTERED » sb ENS 1 « RAW » sb ENS 2 « FILTERED » sb ENS 2 « RAW »

  16. LINK BETWEEN LOCAL SPATIAL AVERAGING AND INCREASE OF SAMPLE SIZE Multiplication by a low-pass spectral filter  Local spatial averaging (convolution) latitude Ng=9  The ensemble size N is MULTIPLIED(!) by a number Ng of gridpoint samples. If N=6 and Ng=9, then the total sample size is N x Ng = 54. • The 6-member filtered estimate is as accurate as a 54-member raw estimate, under a local homogeneity asumption. longitude

  17. DOES SPATIAL AVERAGING OF VARIANCES HAVE AN IMPACT IN THE (VERY) END ? (Raynaud 2008) Ensemble Var at Météo-France (N=6; operational since July 2008) with RAW sb with FILTERED sb obs-analysis obs-guess

  18. Validation withspatially averaged innovation-based variances

  19. Spatial filtering of innovation-based sigmab’s (Lindskog et al 2006) « Raw » innovation-based sigmab’s « Filtered » innovation-based sigmab’s • Some relevant geographical variations (e.g. data density effects), especially after spatial averaging.

  20. Innovation-based sigmab estimate(Desroziers et al 2005) cov( H dx , dy ) ~ H B HT • This can be calculated for a specific date, to examine flow-dependent features, but then the local sigmab is calculated from a single error realization ( N = 1 ) ! • Conversely, if we calculate local spatial averages of these sigmab’s, the sample size will be increased, and comparison with ensemble can be considered.

  21. Innovation-based sigmab’s « of the day » HIRS 7 (28/08/2006 00h) before spatial averaging after spatial averaging (with a radius of 500 km) (Desroziers 2006) • After spatial averaging, some geographical patterns can be identified. Can this be compared with ensemble estimates ?

  22. Validation of ensemble sigmab’s « of the day » HIRS 7 (28/08/2006 00h) Ensemble sigmab’s « Innovation-based » sigmab’s • Spatial averaging makes the two estimates easier to compare and to validate.

  23. Spatial averaging of correlations

  24. Spatial structure ofraw correlation length-scale field RAW L( eb ) = 1/(-2 d²cor/ds²)s=0 « TRUTH » N = 10 Sampling noise : artificial small scale variations. (Pannekoucke et al 2007)

  25. Raw ensemble correlationlength-scale field « of the day » Geographical patterns are difficult to identify, due to sampling noise (N = 6).

  26. Spatial structure ofraw correlation length-scale field N = 5 Reduction of small scale sampling noise, when the ensemble size increases. N = 15 N = 30 N = 60 Sampling noise ~ relatively small scale. • Use spatial filtering. ex : wavelets. (Pannekoucke 2008)

  27. Wavelet diagonal modelling of B(Fisher 2003, Pannekoucke et al 2007) It amounts to local spatial averages of correlation functions cor(x,s): corW(x,s) ~ Sx’ cor(x’,s) F(x’,s) with scale-dependent weighting functions F : small-scale contributions to correlation functions are averaged over smaller regions than large-scale contributions.

  28. Wavelet filtering of correlation functions RAW WAVELET Wavelet approach : sampling noise is reduced, leading to a lesser need of Schur localization. N = 10 (Pannekoucke et al 2007)

  29. Impact of wavelet filteringon analysis quality homogeneous N = 10 raw wavelet Schur Length Wavelet approach : sampling noise is reduced, and there is a lesser need of Schur localization. (Pannekoucke et al 2007)

  30. Wavelet filtering offlow-dependent correlations Anisotropic wavelet based correlation functions ( N = 12 ) Synoptic situation (geopotential near 500 hPa) (Lindskog et al 2007, Deckmyn et al 2005)

  31. Wavelet filtering ofcorrelations « of the day » Raw length-scales Wavelet length-scales N = 6 (Fisher 2003, Pannekoucke et al 2007)

  32. Conclusions • Spatial structures of signal and sampling noise tend to be different. • This leads to « optimal » spatial averaging/filtering techniques (as in data assimilation). • The increase of sample size reduces the estimation error (thus it helps to use smaller (high resolution) ensembles). • Useful for ensemble-based covariance estimation, but also for validation with flow-dependent innovation-based estimates.

  33. Perspectives • Make a bridge with similar/other filtering techniques. ex : spatial BMA technique in probab. forecasting (Berrocal et al 2007). ex : ergodic space/time averages in turbulence. ex : local time averages of ensemble covariances (Xu et al 2008). • The 4D nature of atmosphere & covariance estimation may suggest 4D filtering techniques (instead of 2D currently). • There may be a natural path towards « a data assimilation/filtering » of ensemble- & innovation-based covariances, in order to achieve an optimal covariance estimation ?

  34. Thank you for your attention !

More Related