1 / 61

Data Assimilation in Meteorology and Oceanography

Data Assimilation in Meteorology and Oceanography. Michael Ghil Ecole Normale Supérieure, Paris, and University of California, Los Angeles. Joint work with Dmitri Kondrashov, UCLA, and many others: please see http://www.atmos.ucla.edu/tcd/. Outline. Data in meteorology and oceanography

fflowers
Download Presentation

Data Assimilation in Meteorology and Oceanography

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Assimilation inMeteorology and Oceanography Michael Ghil Ecole Normale Supérieure, Paris, and University of California, Los Angeles Joint work with Dmitri Kondrashov, UCLA, and many others: please see http://www.atmos.ucla.edu/tcd/

  2. Outline • Data in meteorology and oceanography - in situ & remotely sensed • Basic ideas, data types, & issues • how to combine data with models • transfer of information - between variables & regions • stability of the fcst.–assimilation cycle • filters & smoothers • Parameter estimation - model parameters - noise parameters – at & below grid scale • Subgrid-scale parameterizations - deterministic (“classic”) - stochastic – “dynamics” & “physics” • Novel areas of application - space physics - shock waves in solids - macroeconomics • Concluding remarks

  3. Main issues • The solid earth stays put to be observed, the atmosphere, the oceans, & many other things, do not. • Two types of information: - direct  observations, and - indirect  dynamics (from past observations); both have errors. • Combinethe two in (an) optimal way(s) • Advanced data assimilation methods provide such ways: - sequential estimation  the Kalman filter(s), and - control theory  the adjoint method(s) • The two types of methods are essentially equivalent for simple linear systems (the duality principle)

  4. Main issues (continued) • Their performance differs for large nonlinear systems in: - accuracy, and - computational efficiency • Study optimal combination(s), as well as improvements over currently operational methods (OI, 4-D Var, PSAS).

  5. Outline Data in meteorology and oceanography - in situ & remotely sensed • Basic ideas, data types, & issues • how to combine data with models • filters & smoothers - stability of the fcst.-assimilation cycle • Parameter estimation - model parameters - noise parameters – at & below grid scale • Subgrid-scale parameterizations - deterministic (“classic”) - stochastic – “dynamics” & “physics” • Novel areas of application - space physics - shock waves in solids - macroeconomics • Concluding remarks

  6. Atmospheric data Drifting buoys: Ps – 267 Total no. of observations = 0(105) scalars per 12h–24h • 0(102 ) observations/[(significant d-o-f) x (significant t)] Bengtsson, Ghil & Källén (eds.): Dynamic Meteorology, Data Assimilation Methods (1981) Polar orbiting satellites: T – 5x2048 Cloud-drift: V – 2x2259 Aircraft: V – 2x1100 Balloons : V – 2x581x10 Radiosondes : T, V - 3x749x10 Ship & land surface: Ps, Ts , Vs – 4x3446

  7. Observational network Quality control – preliminary & as part of the assimilation cycle

  8. Ocean data – past Total no. of oceanographic observations/met. ob’sns = O(10–4) for the past; & = O(10–1) for the future : Syd Levitus (1982).

  9. Ocean data – present & future Altimetry  sea level; scatterometry  surface winds & sea state; acoustic tomography  temperature & density; etc. Courtesy of Tony Lee, JPL

  10. Space physics data Space platforms in Earth’s magnetosphere

  11. Basic ideas of data assimilation and sequential estimation - I Simple illustration Want to estimate u - temperature of this room, based on the readings u1 and u2 of the two thermometers. Estimate û= 1u1 + 2u2 Interpretation will be: u1 = uf - first guess (of numerical forecast model) u2 = uo - observation(R/S, satellite, etc.) û = ua - objective analysis

  12. Basic ideas of data assimilation and sequential estimation - II If u1 and u2 are unbiased, and û should be unbiased, then 1 + 2 = 1, so one can write û = u1 + 2(u2 - u1): updating (sequential) If u1 and u2 are uncorrelated, and have A1 =1–2, A2 =2–2: known standard deviations, Then the minimum variance estimator(*) is û = u1 + A2 /(A2 - A1) (u2 - u1) and its accuracy is  = (A1 + A2) ≥ max {A1, A2} * BLUE = Best Linear Unbiased Estimator

  13. Kalman Filter - I

  14. Kalman Filter - II

  15. Kalman Filter - III

  16. Kalman Filter - IV

  17. Basic concepts: barotropic model Shallow-water equations in 1-D, linearized about (U,0,), fU = – y U = 20 ms–1, f = 10–4s–1,  = gH, H  3 km. PDE system discretized by finite differences, periodic B. C. Hk: observations at synoptic times, over land only. Ghil et al. (1981), Cohn & Dee (Ph.D. theses, 1982 & 1983), etc.

  18. Trade-off between variables • Some variables are observed, • others are not. Wind info. is better (here) than height info. Height obs’d Observing System Simulation Experiments (OSSE) Wind obs’d Identical twins vs. real observations

  19. Conventional network Relative weight of observational vs. model errors P∞ = QR/[Q + (1 – 2)R] (a) Q = 0 P∞ = 0 (b) Q ≠ 0  (i), (ii) and (iii): • “good” observations R << Q P∞ ≈ R; (ii)“poor” observations R >> QP∞ ≈ Q/(1 – 2); (iii) always (provided 2 < 1) P∞ ≤ min {R, Q/(1 – 2)}.

  20. {6h fcst} - {conventional (NoSat)} Advection of information b) {“first guess”} - {FGGE analysis} Upper panel (NoSat): Errors advected off the ocean 300 {“first guess”} - {FGGE analysis} Lower panel (Sat): Errors drastically reduced, as info. now comes in, off the ocean 300 Halem, Kalnay, Baker & Atlas (BAMS, 1982)

  21. Evolution of DA – I Transition from “early”to “mature” phase of DA in NWP: • no Kalman filter Ghil et al., 1981(*) • no adjoint Lewis & Derber (Tellus, 1985); Le Dimet & Talagrand (Tellus, 1986) (*) Bengtsson, Ghil & Källén (Eds., 1981), Dynamic Meteorology: Data Assimilation Methods. M. Ghil & P. M.-Rizzoli(Adv. Geophys., 1991).

  22. Evolution of DA – II Cautionary note: “Pantheistic” view of DA: • variational ~ KF; • 3- & 4-D Var ~ 3- & 4-D PSAS. Fashionable to claim it’s all the same but it’s not: • God is in everything, • but the devil is in the details. M. Ghil & P. M.-Rizzoli (1991, Adv. Geophys.)

  23. Outline • Data in meteorology and oceanography - in situ & remotely sensed  Basic ideas, data types, & issues • how to combine data with models • stability of the fcst.–assimilation cycle • filters & smoothers • Parameter estimation - model parameters - noise parameters – at & below grid scale • Subgrid-scale parameterizations - deterministic (“classic”) - stochastic – “dynamics” & “physics” • Novel areas of application - space physics - shock waves in solids - macroeconomics • Concluding remarks

  24. Error components inforecast–analysis cycle The relative contributions to error growth of • analysis error • intrinsic error growth • modeling error (stochastic?)

  25. Assimilation of observations: Stability considerations Free-System Dynamics(sequential-discrete formulation): Standard breeding forecast state; model integration from a previous analysis Corresponding perturbative (tangent linear) equation Observationally Forced System Dynamics(sequential-discrete formulation): BDAS If observations are available and we assimilate them: Evolutive equation of the system, subject to forcing by the assimilated data Corresponding perturbative (tangent linear) equation, if the same observations are assimilated in the perturbed trajectories as in the control solution • The matrix (I – KH) is expected, in general, to have a stabilizing effect; • the free-system instabilities, which dominate the forecast step error growth, can be reduced during the analysis step. Joint work with A. Carrassi, A. Trevisan & F. Uboldi

  26. Stabilization of the forecast–assimilation system – I Assimilation experiment with a low-order chaotic model • Periodic 40-variable Lorenz (1996) model; • Assimilation algorithms: replacement (Trevisan and Uboldi, 2004), replacement + one adaptive obs’n located by multiple replication (Lorenz, 1996), replacement + one adaptive obs’n located by BDAS and assimilated by AUS (Trevisan & Uboldi, JAS, 2004). Trevisan & Uboldi (JAS, 2004)

  27. Stabilization of the forecast–assimilation system – II Assimilation experiment with an intermediate atmospheric circulation model • 64-longitudinal x 32-latitudinal x 5 levels periodic channel QG-model (Rotunno & Bao, 1996) • Perfect-model assumption • Assimilation algorithms: 3-DVar (Morss, 2001); AUS (Uboldi et al., 2005; Carrassi et al., 2006) Observational forcing  Unstable subspace reduction  Free System Leading exponent: max ≈ 0.31 days–1; Doubling time ≈ 2.2 days; Number of positive exponents: N+ = 24; Kaplan-Yorke dimension ≈ 65.02.  3-DVar–BDAS Leading exponent: max ≈ 6x10–3 days–1;  AUS–BDAS Leading exponent: max ≈ – 0.52x10–3 days–1

  28. Outline • Data in meteorology and oceanography - in situ & remotely sensed  Basic ideas, data types, & issues • how to combine data with models • stability of the fcst.–assimilation cycle • filters & smoothers • Parameter estimation - model parameters - noise parameters – at & below grid scale • Subgrid-scale parameterizations - deterministic (“classic”) - stochastic – “dynamics” & “physics” • Novel areas of application - space physics - shock waves in solids - macroeconomics • Concluding remarks

  29. The main products of estimation(*) • Filtering (F) – “video loops” • Smoothing (S) – full-length feature “movies” • Prediction (P) – NWP, ENSO Distribute all of this over the Web to • scientists, and the • “person in the street” (or on the information superhighway). In a general way: Have fun!!! (*)N. Wiener (1949, MIT Press)

  30. Kalman smoother For a fixed interval, weak constrained 4-D Var is equivalent to the sequential (“Kalman”) smoother. Cohn, Sivakumaran & Todling (MWR, 1994)

  31. Smoothing vs. Filtering: The Backward Sequential Smoother (BSS) • A “smoother” is smoother than a filter. • But which smoother is - smoothest - cheapest - easiest to implement? Joint work with T. M. Chin, J. B. Jewell, & M. J. Turmom, JPL

  32. EnKF, RPF, MCMC and the BSS The BSSretrospectively updates a set of weights for ensemble members. It can • work with eitherEnKF- or RPF-generated ensembles; • is relatively inexpensive; and • works well for highly nonlinear, illustrative examples: - the double-well potential, & - the Lorenz (1963) model.

  33. BSS Performance for the Lorenz (1963) System Data: x1 and x3, every t = 0.5 Upper panel: RPF vs. smoother Smoother follows ob’ns () better inx1and is more realistic inx2. Lower panel: RPFvs. EnKF, & filter(- - -)vs. smoother(----) Smoother better than filter, & EnKF better than RPF for very small ensemble size N, but RPF takes over as N increases.

  34. Outline • Data in meteorology and oceanography - in situ & remotely sensed • Basic ideas, data types, & issues • how to combine data with models • filters & smoothers - stability of the fcst.-assimilation cycle  Parameter estimation - model parameters - noise parameters – at & below grid scale • Subgrid-scale parameterizations - deterministic (“classic”) - stochastic – “dynamics” & “physics” • Novel areas of application - space physics - shock waves in solids - macroeconomics • Concluding remarks

  35. Parameter Estimation a) Dynamical model dx/dt = M(x, ) + (t) yo = H(x) + (t) Simple (EKF) idea – augmented state vector d/dt = 0, X = (xT, T)T b) Statistical model L() = w(t), L – AR(MA) model,  = (1, 2, …. M) Examples: 1) Dee et al. (IEEE, 1985) – estimate a few parameters in the covariance matrix Q = E(, T); also the bias <> = E; 2)POPs - Hasselmann (1982, Tellus); Penland (1989, MWR; 1996, Physica D); Penland & Ghil (1993, MWR) 3) dx/dt = M(x, ) + : Estimate both M & Q from data (Dee, 1995, QJ), Nonlinear approach: Empirical mode reduction (Kravtsov et al., 2005, Kondrashov et al., 2005)

  36. Estimating noise – I 1 Q1 = Qslow , Q2 = Qfast , Q3 =0; R1 = 0, R2 = 0, R3 =R; Q = ∑ iQi; R = ∑ iRi ; (0) = (6.0, 4.0, 4.5)T; Q(0) = 25*I. Dee et al. (1985, IEEE Trans.Autom. Control, AC-30) estimated 2 true ( =1) 3 Poor convergence for Qfast?

  37. Estimating noise – II 1 Same choice of (0), Qi , and Ribut 1 0.8 0  (0) = 25 *0.8 1 0   0 0 1  Dee et al. (1985, IEEE Trans.Autom. Control, AC-30) 2 estimated true ( = 1) 3 Good convergence for Qfast!

  38. Sequential parameter estimation • “State augmentation” method – uncertain parameters are treated as additional state variables. • Example: one unknown parameter  • The parameters are not directly observable, butthe cross-covariances drive parameter changes from innovations of the state: • Parameter estimation is always a nonlinear problem, even if the model is linear in terms of the model state: use Extended Kalman Filter (EKF).

  39. Parameter estimation for coupled O-A system Forecast using wrong Forecast using wrong and s • Intermediate coupled model (ICM: Jin & Neelin, JAS, 1993) • Estimate the state vector W = (T’, h, u, v), along with the coupling parameter  and surface-layer coefficient s by assimilating data from a single meridional section. • The ICM model has errors in its initial state, in the wind stress forcing & in the parameters. • M. Ghil (1997, JMSJ); Hao & Ghil (1995, Proc. WMO Symp. DA Tokyo); Sun et al. (2002, MWR). • Current work with D. Kondrashov, J.D. Neelin, & C.-j. Sun. Reference solution Assimilation result Reference solution Assimilation result

  40. Convergence of parameter value Reference value:  = 0.76; Initial model (“wrong”) value:  = 0.6

  41. Outline • Data in meteorology and oceanography - in situ & remotely sensed • Basic ideas, data types, & issues • how to combine data with models • stability of the fcst.–assimilation cycle • filters & smoothers • Parameter estimation - model parameters - noise parameters – at & below grid scale • Subgrid-scale parameterizations - deterministic (“classic”) - stochastic – “dynamics” & “physics”  Novel areas of application - space physics - shock waves in solids - macroeconomics • Concluding remarks

  42. Parameter Estimation for Space Physics – I Daily fluxes of 1MeV relativistic electrons in Earth’s outer radiation belt (CRRES observations from 28 August 1990) Kp - index of solar activity (external forcing) Joint work with D. Kondrashov, Y. Shprits, & R. Thorne, UCLA; R. Friedel & G. Reeves, LANL

  43. Parameter estimation for space physics – II HERRB-1D code (Y. Shprits) – estimating phase space density f and electron lifetime L: Different lifetime parameterizations for plasmasphere – out/in: Lo=/Kp(t);Li=const. What are the optimallifetimes to match the observations best?

  44. Parameter estimation for space physics – III Daily observations from the “truth” — Lo = /Kp,  = 3, and LI = 20 — are used to correct the model’s “wrong” parameters, = 10 and LI = 10. The estimated error tr(Pf) ≈ actual When the parameters’ assumed uncertainty is large enough, their EKF estimates converge rapidly to the “truth”.

  45. Deterministic parametrization - I • Hierarchy of climate model:from 0-D EBMs to 3-D GCMs • None resolves all relevant processes on all the scales of motion; • “Parametrization”= representation of unresolved processes in terms of resolved variables. • GCMs, as well as many intermediate, 2-D models, have • “dynamics”  “physics,” i.e., • fluid dynamics (adiabatic, conservative, Hamiltonian) + radiation, clouds, surface processes, precipitation, etc. (forcing & dissipation); • both are only resolved up to a given trancation (finite differences, finite elements, spectral, etc.); • much of the “physics” has characteristic scales that are not resolved.

  46. Deterministic parametrization - II Slow and fast variables • w = (uT,vT)T • u~ large-scale, slow; v~ small-scale, fast • y = x, s = t • (S)du/dt=(u, v), u = u(y, s) • (F)dv/dt=g(u,v),v = v(x, t) • Equations like (S)(F) appear in slow manifold theory, “initialization,” and “parametrization” of small scales.

  47. Deterministic parametrization - III • General idea: (i) Solve (F) at fixed u(y, so) = uo(y) to get (O) v=v(x, t; uo) • then extend to an “adiabatic” solution of (ii) (A)v = v(x, t; u(y, s)).Substitute (A) into (S), to yield (SA) = du/dt=(u(y, s),v(x, t; u(y, s)) • Use nonlinear averaging, on the fast & small scales, • applied to (SA)to get the net effect of these scales on the evolution of the slow & large ones: • (P)du/dt= (u(y, s); vF(y, s)).

  48. Stochastic parametrization • Let’s keep to additive, white, Gaussian noise: (S') du/dt=f(u,v)+ ; =(y, s)  QS=cov(, `) (F') dv/dt=g(u,v); =(x, t) QF=cov(, `) • We proceed in an analogous fashion to get (Q') v = v (x, t; uo, ), = {(x, ): -<<t} and then the “adiabatic” evolution (A') v = v(x, t; u(y, s; ), ),  = {(y, ):-<<s}. • Substituting now (A') into (S') will yield, if we’re lucky: (P’) For luck, we need • Nonlinear averaging procedure: Lie series? • Parameter estimation procedure: EKF? • Explicit noise generation – yes/no?

  49. MTV Majda, Timofeyev & Vanden Eijnden (1999, PNAS; 2001, CPAM)

  50. Outline • Data in meteorology and oceanography - in situ & remotely sensed • Basic ideas, data types, & issues • how to combine data with models • stability of the fcst.–assimilation cycle • filters & smoothers • Parameter estimation - model parameters - noise parameters – at & below grid scale • Subgrid-scale parameterizations - deterministic (“classic”) - stochastic – “dynamics” & “physics”  Novel areas of application - space physics - shock waves in solids - macroeconomics • Concluding remarks

More Related