Ensemble Data Assimilation and Prediction: Applications to Environmental Science

1. Ensemble Data Assimilation and Prediction:Applications to Environmental Science Dusanka Zupanski CIRA/Colorado State University Fort Collins, Colorado Presentation at EWHA Department of Environmental Science and Engineering May 26, 2005 • Collaborators • M. Zupanski, L. Grasso, Scott Denning Group (Colorado State University) • A. Y. Hou, S. Zhang (NASA/GMAO) • M. DeMaria (NOAA/NESDIS) • Seon Ki Park (EWHA Womans University) Research supported by NOAA Grant NA17RJ1228, NASA Grants 621-15-45-78, NAG5-12105 and NNG04GI25G, and DoD Grant DAAD19-02-2-0005. Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

2. OUTLINE • Introduction to ensemble data assimilation methods • Maximum Likelihood Ensemble Filter (MLEF) • State augmentation approach for model error and parameter estimation • MLEF applications • Conclusions and future work Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

3. Forecast error Covariance Pf (ensemble subspace) Observations First guess DATA ASSIMILATION Analysis error Covariance Pa (ensemble subspace) Optimal solution for model state x=(T,u,v,q, Te,ue,ve,qe, ,,) ENSEMBLE FORECASTING Introduction: What is Ensemble Data Assimilation? Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

4. Maximum Likelihood Ensemble Filter (MLEF)(Zupanski 2005; Zupanski and Zupanski 2005) - Change of variable (preconditioning) - C is information matrix of dim Nens Nens - ziare columns of Z - pif and pia are columns of Pf and Pa - Observations vector of dim Nobs - Model state vector of dim Nstate >>Nens - Control vector in ensemble space of dim Nens - Dynamical forecast model - Observation operator Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

5. VARIATIONAL MLEF Ideal Hessian Preconditioning Milija Zupanski, CIRA/CSU ZupanskiM@CIRA.colostate.edu

6. Non-Gaussian Gaussian PDF(x) PDF(x) x x xmode xmean xmode = xmean MODE vs. MEAN MLEF involves an iterative minimization of functional J=>xmode Minimum variance methods (EnKF) calculate ensemble mean =>xmean Different results expected for non-Gaussian PDFs Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

7. Wide Range of Applications • Data assimilation • Ensemble forecasting • Parameter estimation • Model error estimation • Targeted observations • Information content of observations • Advanced parallel computing • Predictability, nonlinear dynamics • Any phenomenon with predictive model and observations: Weather, Climate, Ocean, Ecology, Biology, Geology, Chemistry, Cosmology, …

8. ASSIMILATION Locations of min and max centers are much improved in the experiment with assimilation. DATA ASSIMILATION application: CSU-RAMS non-hydrostatic model: Total humidity mixing ratio (level=200m, Nens=50, Nstate=54000) TRUTH NO ASSIMILATION Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

9. MODEL ERROR AND PARAMETER ESTIMASTIONUsing state augmentation approach (Zupanski and Zupanski 2005) - Forecast model for standard (non-augmented) model state x - Forecast model for model error (bias) b - Forecast model for empirical parameters  Define augmented state vector w=(x,b, ) and use augmented forecast model F to update w from time step n-1 to n as Minimize the following cost function J Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

10. PARAMETER ESTIMATION, KdVB model(Korteweg-de Vries-Burgers model) True parameter recovered. Improved innovation statistics.

11. BIAS ESTIMATION, KdVB model NEGLECT BIAS BIAS ESTIMATION (vector size=101) BIAS ESTIMATION (vector size=10) NON-BIASED MODEL It is beneficial to reduce degrees of freedom of the model error.

12. BIAS ESTIMATION, KdVB model Augmented analysis error covariance matrix is updated in each data assimilation cycle. It includes cross-covariance between the initial conditions (IC) error and model error (ME). Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

13. Stddev(e)=1.e-3 Gaussian framework works only for small observation errors Impact of Lognormal observation errors:Analysis RMS errors, CSU-SWM Stddev(e)=1.e-2 Lognormal framework works for all error magnitudes Milija Zupanski, CIRA/CSU ZupanskiM@CIRA.colostate.edu

14. Gaussian framework Lognormal framework Impact of Lognormal height observation errors:Innovation histogram Stddev(e)=1.e-3 Stddev(e)=1.e-2 Height innovations Generalized non-Gaussian framework can handle Gaussian, Lognormal, or mixed PDF errors !

15. INFORMATION CONTENT ANALYSIS GEOS-5 column model Information matrix Degrees of freedom (DOF) for signal Shannon information content, or entropy reduction (used for quantifying predictability) Inadequate Pf at the beginning of data assimilation New observed information

16. Ongoing work • Ensemble data assimilation and control theory (NSF) • Apply MLEF with CSU global shallow-water model, constructed on a twisted icosahedral grid • Non-Gaussian framework • Non-derivative minimization methods • Precipitation assimilation and moist processes (NASA) • Apply MLEF with NASA GEOS-5 column precipitation model • Model errors • Empirical parameter estimation • GOES-R Risk Reduction (NOAA/NESDIS) • Evaluate the impact of GOES-R measurements in applications to severe weather and tropical cyclones • Information content of GOES-R measurements

17. Ongoing work (continued) • NCEP Global Forecasting System model and real data (NOAA/THORPEX) • Apply MLEF with GFS and compare with other EnKF algorithms • Compare (and combine) the conditional mean and mode PDF estimates • Develop and test double-resolution MLEF • Carbon Cycle data assimilation (NASA) • Apply MLEF with fully coupled SiB-CASA-RAMS atmospheric-carbon-biomass model • Assimilate carbon concentration globally and locally • Empirical parameter estimation in carbon models • Microscale ensemble data assimilation (CGAR-DoD) • Apply MLEF with Univ. Purdue non-hydrostatic model • Assimilation of real data • Collaboration with the Army Research Laboratory

18. Future Research Directions Development of a fully non-Gaussian algorithm • Allow for non-Gaussian state variable components • Generalized algorithm with a list of PDFs • Model bias and parameter estimation • Improve prediction models by learning about its errors and uncertainties • Develop as a probabilistic tool for new model development • Information content analysis • Value added of new observations (e.g., GPM, CloudSat, GOES-R, OCO) • Cross-over the existing scientific boundaries • Apply probabilistic assimilation/prediction to different science disciplines (e.g., atmospheric, oceanic, ecological, hydrological sciences) • General, adaptive, algorithmically simple algorithm opens new possibilities

19. Literature Anderson, J. L. 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131, 634-642. Bishop, B., Etherton, J. and Majmudar, S. J. 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420-436. Cohn, S.E., 1997: An introduction to estimation theory. J. Meteor. Soc. Japan, 75, 257-288. Evensen, G. 2003: The Ensemble Kalman Filter: theoretical formulation and practical implementation. Ocean Dynamics, 53, 343-367. Fletcher, S. J., and M. Zupanski, 2005: A framework for data assimilation which allows for non-Gaussian errors. Submitted to Proc. Royal Soc. of London A. [ftp://ftp.cira.colostate.edu/milija/papers/Steven_nongauss.pdf] Hamill, T.M., and C. Snyder, 2000: A hybrid ensemble Kalman filter-3D variational analysis scheme. Mon. Wea. Rev., 128, 2905-2919. Houtekamer, P.L., H.L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an Ensemble Kalman Filter: Results with real observations. Mon. Wea. Rev., 133, 604-620. Jazwinski, A.H., 1970: Stochastic processes and filtering theory. Academic Press, New York, 376 pp. Ott, E., Hunt, B. R., Szunyogh, I., Zimin, A. V., Kostelich, E. J., Corazza, M., Kalnay, E., Patil, D. J. and Yorke, J. A. 2004: A Local Ensemble Kalman Filter for Atmospheric Data Assimilation. Tellus, 56A, No. 4, 273-277. Reichle, R. H., McLaughlin, D. B. and Entekhabi, D. 2002a: Hydrologic data assimilation with the Ensemble Kalman Filter. Mon. Wea. Rev., 130, 103-114. Snyder, C., and Zhang, F. 2003: Assimilation of simulated Doppler radar observations with an ensemble Kalman filter. Mon. Wea. Rev., 131, 1663-1677. Tippett, M., J.L. Anderson, C.H. Bishop, T.M. Hamill, and J.S. Whitaker, 2003: Ensemble square-root filters. Mon. Wea. Rev., 131, 1485-1490. Whitaker, J. S., and Hamill, T. M. 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130,1913-1924. Zupanski, M. 2005: Maximum Likelihood Ensemble Filter: Theoretical Aspects. Mon.Wea.Rev., in print. [ftp://ftp.cira.colostate.edu/milija/papers/MLEF_MWR.pdf] Zupanski, D. and M. Zupanski, 2005: Model error estimation employing ensemble data assimilation approach. Submitted to Mon. Wea. Rev. [ftp://ftp.cira.colostate.edu/Zupanski/manuscriptss/MLEF_model_err.Feb2005.pdf] Review paper Review paper General Review paper

21. Summary of basic characteristics of the MLEF • Developed using ideas from -Variational data assimilation(3DVAR, 4DVAR) -Iterated Kalman Filters -Ensemble Transform Kalman Filter (ETKF, Bishop et al. 2001) • Fully non-linear approach. Adjoint models are not needed. • Non-derivative minimization (finite difference is used instead of first derivative). • It can use Gaussan and non-Gaussian errors • Calculates optimal estimatesof: - model state variables (e.g., air temperature, soil moisture, tracer fluxes) - empirical parameters (e.g., diffusion coefficient, drought stress) - model error (bias) - boundary conditions error (lateral, top, bottom boundaries) • Calculates uncertainty of all estimates (in terms of Pa and Pf) • Calculates information measures (e.g., degrees of freedom for signal, entropy reduction) Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

22. DATA ASSIMILATION application: CSU-RAMS atmospheric non-hydrostatic model • Hurricane Lili case • 35 1-h DA cycles: 13UTC 1 Oct 2002 – 00 UTC 3 Oct • 30x20x21 grid points, 15 km grid distance (in the Gulf of Mexico) • Control variable: u,v,w,theta,Exner, r_total (dim=54000) • Model simulated observations with random noise (7200 obs per DA cycle) • Nens=50 • Iterative minimization of J (1 iteration only) RMS errors of the analysis (control experiment without assimilation) Hurricane entered the model domain. Impact of assimilation more pronounced. Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu