1 / 21

Outline

tess
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2004 SIAM Annual MeetingMinisymposium on Data Assimilation and Predictability for Atmospheric and Oceanographic Modeling July 15, 2004, Portland, Oregon ISSUES IN FURTHER DEVELOPMENT OFENSEMBLE DATA ASSIMILATIONMilija ZupanskiCooperative Institute for Research in the AtmosphereColorado State UniversityFort Collins, CO 80523-1375ZupanskiM@CIRA.colostate.edu

  2. Outline • Probabilistic analysis-prediction - ensemble framework • Gaussian Probability Density Function (PDF) framework - non-Gaussian PDFs - nonlinearity • Maximum Likelihood Ensemble Filter (MLEF) • Model Errors

  3. Why ensemble data assimilation ? • Analysis-prediction problem is probabilistic • Inherent uncertainties in observed and predicted values: - Observation errors - Model errors - Turbulence, Convection • Kolmogorov equation: - Transport of Probability Density Function (PDF) - General mathematical framework for analysis-prediction • Chaotical Atmosphere/Ocean/Land processes: • - Existence of low-dimensional attractor subspace suggests the need for ‘likelihood’, rather than deterministic knowledge of prediction • Highly nonlinear processes and interactions in real atmosphere/ocean • - Ensemble DA methodologies are best equipped to handle nonlinearities • Practical aspects • - Parallel computing, code development

  4. Forward Kolmogorov Equation p– probability density function (PDF); f – dynamical model; g– stochastic forcing (model error) Prediction Data Assimilation • Prediction: Estimate of the forecast PDF • Data Assimilation: Estimate of the initial PDF

  5. Implications of Kolmogorov Equation Framework • THERE IS A SINGLEPROBABILISTIC ANALYSIS-PREDICTION SYSTEM Current systems: - only weak coupling betweenanalysisandprediction - modeled forecast PDF information in data assimilation - practical DA algorithms estimate only a single PDF parameter (e.g., PDF mode) - analysis PDF estimate is commonly NOT produced • New systems: • - fully coupled: complete feedback between prediction and analysis • - estimate of: (i) analysis PDF, and (ii) forecast PDF • - possibility to estimate various PDF parameters: mode, mean, covariance, . . .

  6. What do we want from PDF? • Likelihood of an event occurring - optimal PDF parameter estimate - uncertainty of the estimate • PDF parameters - conditional mean - conditional mode - covariance - . . . Gaussian PDF Maxwell PDF • Conditional probability using Bayes formula: Event A: Event B:

  7. Practical limitations of PDF parameter estimation • LARGE NUMBER OF DEGREES OF FREEDOM (DIMENSIONS) - computational burden: memory allocation, efficiency • REDUCING THE NUMBER OF DEGREES OF FREEDOM - statistical sampling of PDF - ensemble framework: span dynamically important (e.g., unstable) subspace • Statistical PDF parameters estimation methods: • Minimum variance:Ensemble mean • - Monte Carlo (ensemble) Kalman Filter (EnKF) – stochastic filters • - Ensemble Square-Root filters (EnSRF)– deterministic filters • Maximum likelihood: Ensemble mode (deterministic control) • - variational data assimilation • - Maximum Likelihood Ensemble Filter (MLEF)

  8. EKF/EnKF/EnSRF as a quadratic optimization process Consider a Gaussian conditional PDF Subject to Form a quadratic cost function: J= - ln(Pr) Pf- forecast error covariance R - observation error covariance H - nonlinear observation operator H - linearized observation operator (Jacobian) y - observation vector x - analysis vector xb - first-guess vector • Search for x (e.g., analysis) which maximizes the conditional probability (e.g., minimizes the cost function)

  9. Linear KF analysis solution (with Gaussian PDF assumption) Maximum likelihood and minimum variance estimates identical for Gaussian PDF (1) One-step solution of quadratic optimization problem: linear H=> step-length a=1 (2) Direct solution of EKF/EnKF/EnSRF: Linear solution framework: EKF, EnKF, EnSRF solution form obtained by assuming linear observation operators

  10. Nonlinearity Issue 1: Observation and model operators are highly nonlinear • - Nonlinear prediction model M used in Pf • - Nonlinear observation operator H used in PfHTand HPfHT • Options: • (1) Use linearform of the solution, combined with nonlinear models in covariance calculation • - current EnKF, EnSRF algorithms • Directly search for nonlinear solutionbyminimizing non-quadratic cost function • - Maximum Likelihood Ensemble Filter (MLEF) • Remaining question: • - How restrictive is the linear form of the KF, EnKF solution ? • - Should nonlinearity of H be included in a more consistent manner ?

  11. Non-Gaussian PDF assumption • Fundamental problem: Inconsistent PDF assumption • - Operators are nonlinear (observation, model), Gaussian assumption violated • - Gaussian assumption known to be incorrect for some variables (e.g., precipitation, clouds, etc.) • - Current mathematical framework used in realistic data assimilation relies heavily on Gaussian PDF assumption (e.g., cost function, PDF) • Need general mathematical framework: Non-Gaussian PDFs A solution: Within the Max Likelihood (MLEF) approach, optimize arbitrary non-Gaussian conditional PDF • Remaining problem: Multi-modal PDFs

  12. Statistical PDF parameters Mean Mode PDF PDF Uni-modal Mean Mode PDF PDF Bi-modal Dynamical state Dynamical state

  13. Maximum Likelihood Ensemble Filter (MLEF): • MLEF developed using ideas from: • Variational data assimilation(3DVAR, 4DVAR) • Iterated Kalman Filters • Ensemble Transform Kalman Filter (ETKF) • Algorithm specifics: • Nonlinear cost function minimization – as in 3DVAR, 4DVAR • Unconstrained minimization, well suited for larger residuals (C-G, LBFGS) • Hessian preconditioning using the ETKF transformation • Major assumption: Inverse Hessian = Analysis error covariance • => satisfactory if solution is close to the minimum References Zupanski, D., and M. Zupanski, 2004: Model error estimation employing ensemble data assimilation approach. Submitted to Mon. Wea. Rev. [Available at ftp://ftp.cira.colostate.edu/milija/papers/MLEF_model_err.pdf] Zupanski, M., 2004: The Maximum Likelihood Ensemble Filter. Theoretical aspects. Submitted to Mon. Wea. Rev.[Available at ftp://ftp.cira.colostate.edu/milija/papers/MLEF_MWR.pdf]

  14. Maximum Likelihood Ensemble Filter (MLEF) • conditional PDF mode by minimization of cost function Korteweg-de Vries-Burgers (KdVB) Equation • Experiment: • Nonlinear advection, dispersion, diffusion • Periodic boundary conditions • Two solitary waves (solitons) • Model domain: 101 grid-points • Observation error: 0.05 units • 10 observations (perfect model + perturbation) • 3 minimization iterations in each MLEF analysis cycle

  15. MLEF data assimilation with KdVB model(quadratic obs operator, 10 ensembles, 10 obs) H(x)=x2 RMS error Analysis error covariance NO OBS MLEF NO MIN MLEF Model dynamics helps in localization of analysis error covariance !

  16. Model errors in Ensemble Data Assimilation (EnsDA) • More important than ever before ! • - Forecast error covariance information relies on model forecasts: • if incorrect, the forecast error covariance is incorrect ! • Model bias, empirical parameters, physics, truncation errors, … • Improve the spread of ensemble forecasts • Optimal estimate of model error • Optimal estimate of model error covariance • Can be used to learn about the sources of model error

  17. Model error estimation • State augmentation approach: • - adopted in MLEF (and NCEP’s Eta 4DVAR) x0 – initial conditions ; b – model bias ; g – empirical parameters Augmented control variable: Augmented error covariance:

  18. Model error estimation – cont. • State augmentation approach: • - initial conditions + model bias x0 – initial conditions ; b – model bias Augmented control variable: Augmented error covariance:

  19. MLEF data assimilation with KdVB model Augmented analysis error covariance matrix Cross-covariance between model bias and initial conditions: Px0,b Auto-covariance for model bias: Pb.b Auto-covariance for initial conditions: Px0,x0 From Zupanski and Zupanski 2004, MWR [Available at ftp://ftp/cira.colostate.edu/milija/MLEF_model_err.pdf]

  20. Conclusions • Unified probabilistic analysis-prediction system is important in addressing the atmospheric and oceanographic issues: - samplingof analysis-prediction PDF (ensemble framework) - complete feed-backbetween ensemble data assimilation and ensemble forecasting • Treatment of nonlinearities of prediction model and observation operator can be improved with cost function minimization (MLEF) • Model errors (bias, empirical parameters) need to be included in realistic ensemble data assimilation applications • Need non-Gaussian PDF framework

  21. Future development • Non-Gaussian PDF framework within MLEF approach - Control theory application - Direct optimization of non-Gaussian conditional PDFs - Nonlinear observation and model operators - Global shallow-water model on geodesic grid - Optimization algorithms, Hessian preconditioning • Applications with NCEP’s Global Forecasting System - Comparison between the conditional mean and conditional mode ensemble data assimilation - Real measurements, operational prediction model - Practical aspects: fine resolution control, coarse resolution ensembles

More Related