Accounting for ensemble variance inaccuracy with Hybrid Ensemble 4D-VAR

1. Accounting for ensemble variance inaccuracy with Hybrid Ensemble 4D-VAR “There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.”— Donald Rumsfeld, 2/12/2002, then U.S. Secretary of Defense Hybrid provides framework for accounting for the inaccuracy of our knowledge of unknown unknowns in environmental state estimation Craig Bishop1, Elizabeth Satterfield2, David Kuhl3, Tom Rosmond4 1Naval Research Laboratory, Monterey, CA 2NRC/Naval Research Laboratory, Monterey, CA 3NRC/Naval Research Laboratory, Washington DC, CA 4Science Application International Corp., Forks, WA.

2. Overview • How is Hybrid-Ensemble-AR different from NAVDAS-AR? • Theoretical justification for Hybrid • Performance of Hybrid at T119L42 control, T47L42 ensemble/inner-loop. Hybrid weights from brute force tuning. We will call this the standard Hybrid. • New theory gives weights directly from archive of (ensemble-variance, innovation) pairs using new theory. (6 regions). We call this the equation Hybrid. • Performance comparison: tuned weights versus theoretical weights. • Summary • Discussion: Hybrid versus LETKF/EnKF/EAKF Note: AR=Accelerated-Representer=4D-VAR-in-obs-space

3. How is ensemble-AR different from NAVDAS-AR?

4. Theoretical justification for Hybrid arises from treatment of the hidden volatility problem Deep trough over NE US Ensemble based uncertainty prediction Hidden volatility: When the error variance (volatility) depends on a flow pattern that is unlikely to repeat itself, it is hidden because one or two realizations of error do not enable an estimation of variance.

5. Replicate Earths for ultimate uncertainty quantification Imagine an unimaginably large number of quasi-identical Earths.

6. Hybrid DA and uncertainty quantification in presence of Hidden Volatility (1D idealization) Note that inverse-gamma ensures that innovation variance is never equal to zero.

7. Quantifying uncertainty in presence of Hidden Volatility (1D idealization)

8. Quantifying uncertainty in presence of Hidden Volatility (1D idealization)

9. Description of Experiment • Cycling analysis from Nov. 20, 2008 to Dec. 31, 2008 • Assimilating only conventional observations (no radiances) • Background error covariance matrix at beginning of DA time window (Pb0) is a combination of 75% Pb0_static and 25% Pb0_ensemble Pb0_hybrid=0.75*Pb0_static+0.25*Pb0_ensemble • 32 member ET ensemble (Bishop&Toth, 99,McLay et al 08) • Model resolution: T119L42 Outer, T47L42 Inner • Skip first 10 days of analysis for spin-up • 5-day forecasts from each analysis • Verification of forecasts with Radiosondes • Comparison with current NAVDAS-AR at same resolution using static covariance model

10. Global: TLM-AR (blue) vs. pb0_a025 (red) Contours filled with red indicate regions where Pb0_hybrid reduced rms error (as measured by radiosondes) by more than 5% relative to Pb0_static. The stronger the tone of red the greater the percentage reduction. Colored blocks indicate the significance of difference with red again signifying superiority of Pb0_hybrid. Hybrid_Ensemble_AR performed significantly better than current system

11. Height: TLM-AR (blue) vs. pb0_a025 (red) Temp: TLM-AR (blue) vs. pb0_a025 (red)

12. Vec-Wind: TLM-AR (blue) vs. pb0_a025 (red) Rel. Humid.: TLM-AR (blue) vs. pb0_a025 (red)

13. Derivation of Hybrid weights from analysis of innovations • Assuming that all of the assumptions of simple model of error variance prediction are satisfied, Hybrid weights for optimal error variance prediction may be derived using appropriate regression formula from a large number of independent (ensemble-variance, innovation) pairs.

14. Comparison of variances from globally tuned Hybrid weights with those from theoretically optimal weights for 6 regions (SH, Tropics, NH all below 400 hPa and then all 3 again above 400 hPa) Average ensemble variance Static variance from Pb0 Hybrid variance (theoretical weights) Hybrid variance (tuned weights)

15. U-wind 850mb Raw 6 hr ET variance exhibits large regions of very small variance. This spot is on the West Coast where radiosondes are present but the ensemble variance is below 2. Equation hybrid variance exhibits a smaller range of variances. The analysis will be able to draw to the observations near the West Coast.

16. Test of ensemble-AR using eq Hybrid • Machine: 8 processor Linux cluster • 32 member ET ensemble • Observations: All conventional obs • Period: 11/20/08 to 12/31/08 (1st week removed from assessment and last 5 days from assessment of 120 hr forecasts) • 6 hr DA cycle.

17. Equation Hybrid Global: TLM-AR (blue) vs. pb0_206E1 (red) Globally, max improvements larger for equation-Hybrid than for standard-Hybrid but area of significant improvement larger for standard Hybrid

18. Standard Hybrid Global: TLM-AR (blue) vs. pb0_a025 (red) Globally, max improvements larger for equation-Hybrid than for standard-Hybrid but area of significant improvement larger for standard Hybrid

19. Height: TLM-AR (blue) vs. pb0_206E1 (red) Equation Hybrid Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid. (tropics particularly) Temp: TLM-AR (blue) vs. pb0_206E1 (red)

20. Height: TLM-AR (blue) vs. pb0_a025 (red) Standard Hybrid Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid. (tropics particularly) Temp: TLM-AR (blue) vs. pb0_a025 (red)

21. Vec-Wind: TLM-AR (blue) vs. pb0_206E1 (red) Equation Hybrid Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid. (tropics particularly) Rel. Humid.: TLM-AR (blue) vs. pb0_206E1 (red)

22. Vec-Wind: TLM-AR (blue) vs. pb0_a025 (red) Standard Hybrid Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid. (tropics particularly) Rel. Humid.: TLM-AR (blue) vs. pb0_a025 (red)

23. Summary • All error variance predictions are inaccurate • When two or more independent error variance predictions are available, linear combination is more accurate than either one individually – hybrid allows such combinations • Brute force tuning of Hybrid weights is very expensive • Analytical theoretical model for error variance prediction has been developed that allows weights to be derived directly from archive of (ensemble-variance, innovation) pairs. • Low-resolution experiments using Navy forecast model indicate that (a) tuned global weights (equation Hybrid) make Hybrid-ensemble-AR - superior to standard-AR (b) regional weights from theory (equation Hybrid) better than tuned global weights.

24. Discussion: Hybrid VAR/AR versus LETKF/EnKF/EAKF • Hybrid still works well when only, say, 2 ensemble members can be generated. • TLM/adjoint are optional in Hybrid 4DVAR/AR but adaptive ensemble covariance localization would be required for long time windows – convenient for coupled models. • LETKF/EnKF/EAKF ensemble covariance localization is problematic for obs that are integrals of the state. • Balance constraints easy to apply in Hybrid, difficult in LETKF/EnKF/EAKF. • As yet, fully consistent ensemble update is only possible using perturbed obs with Hybrid. LETKF and EAKF allow update without perturbed obs. • Ensemble covariance localization: freedom from obs space makes adaptive, spectral and statistical localization either easier or the same.

25. Application to global NWP model 06Z Ensemble based localization moves about 1000 km in 12 hrs. This is >=half-width of a typical LETKF observation volume (~900km).

26. Naval Research Laboratory Marine Meteorology Division Monterey, California Application to global NWP model 18Z Ensemble based localization moves about 1000 km in 12 hrs. This is >=half-width of a typical LETKF observation volume (~900km).

27. Application to global NWP model <vv> Increment 06Z Statistical TLM implied by mobile adaptively localized covariance propagates single observation increment 1000 km in 12 hrs.

28. Application to global NWP model <vv> Increment 18Z Statistical TLM implied by mobile adaptively localized covariance propagates single observation increment 1000 km in 12 hrs.

29. 1-point Covariance (Hodyss, 2008 presentation) Initial Time Final Time Red – True 1-point covariance Green – Non-adaptively localized covariance Blue – Multi-scale DAMES covariance