230 likes | 361 Views
ROMS Framework and Algorithms. Hernan G. Arango IMCS, Rutgers. John L. Wilkin IMCS, Rutgers. Javier Zavala-Garay IMCS, Rutgers. Bruce D. Cornuelle SIO, UCSD. Emanuele Di Lorenzo Georgia Tech. Arthur J. Miller SIO, UCSD. Andrew M. Moore UCSC. ROMS/TOMS European Workshop
E N D
ROMS Framework and Algorithms Hernan G. Arango IMCS, Rutgers John L. Wilkin IMCS, Rutgers Javier Zavala-Garay IMCS, Rutgers Bruce D. Cornuelle SIO, UCSD Emanuele Di Lorenzo Georgia Tech Arthur J. Miller SIO, UCSD Andrew M. Moore UCSC ROMS/TOMS European Workshop Alcala de Henares, Spain, November 7, 2006
ROMS Directory Tree Adjoint Bin Drivers External Include Modules Nonlinear Obsolete Programs SeaIce Representer Tangent Utility Version Compilers Lib makefile src Master ROMS SWAN WRF
ROMS Drivers ocean_control.F #include “cppdefs.h” #if definedAD_SENSITIVITY # include "adsen_ocean.h" #elif definedAFT_EIGENMODES # include "afte_ocean.h" #elif definedFT_EIGENMODES # include "fte_ocean.h" #elif definedFORCING_SV # include "fsv_ocean.h" #elif definedOPT_PERTURBATION # include "op_ocean.h" #elif definedOPT_OBSERVATIONS # include "optobs_ocean.h" #elif definedSO_SEMI # include "so_semi_ocean.h" #elif definedS4DVAR # include "s4dvar_ocean.h" #elif definedIS4DVAR # include "is4dvar_ocean.h" #elif definedW4DPSAS # include "w4dpsas_ocean.h" #elif definedW4DVAR # include "w4dvar_ocean.h" #else # if definedTLM_DRIVER # include "tl_ocean.h" # elif definedRPM_DRIVER # include "rp_ocean.h" # elif definedADM_DRIVER # include "ad_ocean.h" # else # include "nl_ocean.h" # endif #endif master.F #include “cppdefs.h” #if definedAIR_OCEAN # include "air_ocean.h" #elif definedWAVES_OCEAN # include "waves_ocean.h" #else # include "ocean.h" #endif ocean.h #include “cppdefs.h” PROGRAM ocean USE ocean_control_mod, ONLY : initialize USE ocean_control_mod, ONLY : run USE ocean_control_mod, ONLY : finalize #ifdef DISTRIBUTE && defined MPI CALL mpi_init (MyError) CALL mpi_comm_rank (MPI_COMM_WORLD, MyRank, MyError) #endif CALL initialize CALL run CALL finalize #if definedDISTRIBUTE&& definedMPI CALL mpi_finalize (MyError) #endif END PROGRAM ocean
ROMS Adjoint • The Adjoint Model (ADM) of ROMS is exact and defined relative to the L2-norm inner-product • Hand-written using the recipe of Giering and Kaminski (1998) • Two Tangent Linear Models: • Perturbation Tangent Linear Model (TLM): Generalized Stability Theory (GST) Analyses, Strong and Weak Constraint 4DVar • Finite Amplitude Tangent Linear Model (RPM): Indirect Representers, Weak Constraint 4DVar • The TLM is derived by linearizing the Nonlinear Model (NLM) around a small perturbation • The RPM model is derived from the TLM by adding additional terms
Parallelization • Coarse-grained parallelization: horizontal tiles • TheNLM, TLM, andRPM can be run in eithershared-memory (OpenMP) or distributed-memory(MPI) • TheADMcan only be run in distributed-memory(ADMviolates shared-memory collision rules) • Aggregation of variables for MPI communications CALLad_mp_exchange2d (ng,iADM, 3, Istr, Iend, Jstr, Jend,& &LBi, UBi, LBj, UBj,& &NghostPoints, EWperiodic, NSperiodic,& &ad_Zt_avg1,ad_DU_avg1,ad_DV_avg1)
Some Uses of Adjoint Models • Data Assimilation: fit model solutions to data by adjusting initial conditions, boundary conditions and parameters. • Sensitivity Analysis: study the response of the ocean circulation to variations in all physical attributes of the system • Eigenmode Analysis: dynamic modes of variability (TLM normal modes, ADM optimal excitations) • Singular Vectors: stability of the dynamical system (most rapidly growing perturbations) • Stochastic Optimals: most disruptive patterns of ocean forcing • Ensemble Prediction: initial condition perturbations along the most unstable directions of the state space • Adaptive Sampling: design of optimal observational systems
4D Variational Data Assimilation (4DVAR) • Strong Constraint • Conventional (S4DVAR): outer loop,NLM,ADM • Incremental (IS4DVAR): inner and outer loops,NLM,TLM,ADM(Courtier et al., 1994) • Weak Constraint • Indirect Representer Method (W4DVAR): inner andouter loops,NLM,TLM,RPM,ADM (Egbert et al., 1994; Bennett et al, 1997) • Physical Space Statistical Analysis (W4DPSAS):inner and outer loops,NLM,TLM,ADM (Courtier, 1997)
xk = B-1/2(xk + xk-1 – xb) xk = B1/2vk + xk-1 – xb Strong Constraint, Incremental 4DVAR Let’s introduce a new minimization variablev, such that: yielding J(vk) = ½(vk)Tvk + ½(Hxk – dk-1)TO-1(Hxk – dk-1) The gradient ofJin minimization-space, denotedv J, is given by: v J = vk + BT/2HTO-1(Hxk – dk-1) = vk + BT/2x Jo = vk + W-1/2LT/2GS The background-error covariance matrix can be factored as: B = SCS = S(GL1/2W-1/2)(W-1/2LT/2G)S whereSis the background-error standard deviations,Cis thebackground-error correlations which can be factorized asC = C1/2CT/2,Gis the normalization matrix which ensures that the diagonalelements ofCare equal to unity,Lis a 3D self-adjoint filteringoperator, andWis the grid cell area or volume.
Model/Background Error Covariance, B • Use a generalized diffusion squared-root operator (symmetric) as in Weaver et al. (2003): B = S C S = S (G L1/2 W-1/2) (W-1/2 LT/2 G) • The normalization matrix,G, ensure that the diagonalelements of the correlation matrix,C, are equal to unity.They are computed using theexact(expensive) orrandomization(cheaper) methods. • The spatial convolution of the self-adjoint filtering operator,L1/2, is split in horizontal and vertical components anddiscretized bothexplicitlyand implicitly. • The model/background standard deviation matrix,S, iscomputed from long (monthly, seasonal) simulations. • The grid cell area or volume matrix,W-1/2, is assumed to betime invariant.
Strong Constraint, Incremental 4DVAR Compute basic state trajectory and extract model at observations locations Misfit cost function between model (NLM+TLM) and observations Cost function gradient Compute TLM initial conditions using first guess conjugate gradient step size Compute change in cost function Compute TLM initial conditions Using refined conjugate gradient step size Compute NLM new initial conditions (NLM+TLM)
Horizontal Vertical (implicit) Vdecay = 100 m Hdecay = 100 km Model/Background Error Correlation (C)
SSH Temperature EAC EAC Bottom Level Model/Background Error Correlation Normalization Coefficients (G)
Generalized Stability Theory (GST) • Dynamics/sensitivity/stability of flow tonaturally occurring perturbations • Dynamics/sensitivity/stability due toerror or uncertainties in the forecast system • Practical applications: • Ensemble prediction • Adaptive observations • Array design ...
Optimal Perturbations A measure of the fastest growing of all possible perturbations over a given time interval RT(t,0)XR(0,t)u u
Ensemble Prediction • Optimal perturbations / singular vectors and stochastic optimals can also be used to generate ensemble forecasts. • Perturbing the system along the most unstable directions of the state space yields informationabout thefirstandsecondmoments of the probability density function (PDF): • ensemble mean • ensemble spread • Excite with dominant basis vectors
Ensemble Prediction For an appropriate forecast skill measure,s
Temperature along XBT line SSH Observations Temperature along XBT line SSH Assimilation EAC: Incremental 4DVar (IS4DVAR) SSH, SST, XBT Assimilation
EAC: Optimal Perturbations Ritz Eigenvalues + 0-days 10-days SV 2 SV 6 SV 7 SV 8 SV 10
EAC: Ensemble Prediction 1-day forecast 8-days forecast 15-days forecast
Profiling ROMS Kernel: EAC • The NLM is run in the EAC application for 800 time-stepsas standalone driver with the same CPP options as the TLM, RPM, ADM: • TLMis 2.5 times more expensive thanNLM • RPMis 2.6 times more expensive thanNLM • RPMis 3-percent more expensive thanTLM • ADMis 2.8 times more expensive thanNLM • ADMis 13-percent more expensive thanTLM • Linux 2.6.12-15mdksmp #1 SMP Mon Jan 9 23:35:18 MST 2006 x86_64 Dual Core AMD Opteron(tm) Processor 265