ec.gc

1. www.ec.gc.ca Lunch Ouranos 8 juillet 2008 Qu’est-ce que le CRCM5 ? Et quel est son rapport avec GEM-LAM, GEMCLIM et GEM(DM) Bernard Dugas Divisionde la rechercheMétéorologique Environnement Canada

2. www.ec.gc.ca Lunch Ouranos 8 juillet 2008 Contenu • CRCM5 ? Quid ? • GEM(DM) vs GEMCLIM • Vertical Staggering (Vstag) (Nouveau) • Modèle Environnemental Couplé (MEC) (à venir)

3. www.ec.gc.ca Lunch Ouranos 8 juillet 2008 Qu’est-ce que le CRCM5 ? Brièvement… • C’est une version du modèle de prévision météorologique GEM(DM) d’Environnement Canada qui pourra éventuellement être utilisée dans des études de scénarios de changement climatique. Mais auparavant, ce modèle pourra être utilisé par la communauté de recherche sur des plateformes informatiques massivement parallèles telle que ce qui est envisagé avec CLUMEQ2. • Le climat régional pourra être simulé avec le CMRC5 soit avec une approche LAM ou avec une approche de grilles à mailles variables. • Enfin, le CRCM5 pourra être utilisé avec une version native à GEM(DM) des paramètrages physiques (déjà disponible, RPN/CMC) ou bien avec celle utilisée par le CGCM4 du CCCma (en cours de développement, projet CRCMD 4.1.1). À partir de là, il sera possible d’implanter d’autres physiques dans le CRCM5. • Notons qu’une version climat de GEM(DM) est à l’essai à l’UQAM depuis maintenant deux années. Il s’agit de GEMCLIM v_3.3.0 installé sur le « cluster » Linux Sunfire appellé marvin.

4. GEMCLIM GEMCLIM vs GEM(DM) • GEMCLIM: a way of running climate-long simulations with GEM • automatic re-launching multi-month jobs • optional, automatic and large amount of post-processing • (comprehensive set of time averages and variances on pressure- and • model-levels, and time series of frequently used variables) • GEMCLIM: a library (routines, scripts) • GEMCLIM versus GEM(DM) • compared with GEM(DM) 3.3: 92% of 570 dynamics routines are the same • compared with PHY 4.5: 98% of 577 physics routines are the same • “mode-backward” compatible: can run non-climate mode with GEMCLIM • Main differences: • post-processing / diagnostic scripts • size-reducing of pilot files from analyses data on pressure levels GEMCLIM: people involved in development at Dorval M. Desgagné, B. Dugas, P. Vaillancourt, K. Winger (UQAM), A. Zadra (in alphabetical order) + plus several others

5. GEMCLIM GEMCLIM vs GEM(DM) There may come a time when the GEM(DM) and GEMCLIM model versions are fully synchonized. Or not. Be that as it may, as long as Environment Canada supports GEM(DM), there will be a corresponding version of GEMCLIM. Finally, the upcoming version of GEMCLIM (v_4.0.x) which should be available by the end of 2008, can be considered as the first CMRC5 release candidate.

6. GEMCLIM About GEMCLIM • GEMCLIM: raison d’etre • test bed for new code and model development • changes can be implemented faster than in operations • (e.g. CCCma Corr.-K radiation; GCM4 physics in GEM) • code changes may be evaluated with robust statistics • collaboration with climate modelling community • GEMCLIM: documentation and support • GEMCLIM versions 3.2.1 and 3.2.2 are currently documented on • http://collaboration.cmc.ec.gc.ca/science/rpn/gem/gem-climate/ • Version 3.3.0 web site is in preparation. • GEMCLIM: computer time per simulation • “short” 2-year simulation, global uniform at 2º resolution: • as little as 1 day with 16 CPUs on the Dorval AIX clusters • a 41-year simulation, LAM over Europe, 0.22º resolution: • - nearly a month, with 4X the resources above

7. GEMCLIM About GEMCLIM • GEMCLIM: simulations performed at Dorval and UQAM • various simulations to test new versions of model / physics: • e.g. mesoglobal, meso-strato, new radiation, aerosols, geophysical fields, etc. • climate-related projects • (Global) • SGMIP1: 12-year 1987-1998, 0.45 - 1.8 deg SG, 318x226 (core 135x146 NA) • SGMIP2: 26-year 1978-2004, 0.5 - 1.5 deg SG, 304x204 (core 79x110 NA/EU) • 26-year 1978-2004, 1.0 deg UG, 360x180 • And with each new model version, at least one AMIP2 1978-200* global 1.5 deg and • one 2.0 deg reference simulations are run. • (LAM) • ICTS (multiple sets, after finding a problem with the SSTs • NA/EU domains driven by GEMCLIM or ERA40, usually at 0.5 deg • EU ENSEMBLES 41-year at 0.22 deg ERA40 (i.e. current climate only !) • Recently, GEMCLIM configured for CLUMEQ LAM Benchmarks • 1) 45-day runs, 640x592x48 0.125 deg, delt= 450 s, • Dorval/AIX : 10 x 11h with 112 CPUs (7x 4x4) • UQÀM/Linux: 10 x 17h with 84 CPUs (7x12x1) • 2) 45-day runs, 320x296x48 0.25 deg, delt= 900 s • Dorval/AIX : 20 x 2h45 with 64 CPUs (4x 4x4) • UQÀM/Linux: 20 x 4h30 with 56 CPUs (8x 7x1)

8. The Vstag Project, i.e. GEM 4.0.2 - Charney-Phillips vertical staggering in GEM(DM) - New vertical coordinate z=lnp* - Highlights of results Claude Girard, André Plante and Sylvie Gravel : Staggered Semi-Lagrangian Scheme Abdessamad Qaddouri : Non-symmetric Elliptic Solver Stéphane Chamberland : Staggered physics interface Lubos Spacek : Staggered physics Vivian Lee : Staggered Input/Output Michel Desgagné : Staggered Coordination

9. The New equations of GEM vertically discretized on a Charney-Phillips grid Vh w T q,(s) m f’ :8 8: diag diag Boundary Conditions:

10. ln(p/ptop)=ln(p/p)+ln(p/p*)+ln(p*/pT) ln(p/ptop)= q + B s + z

11. Virtual Virtual

13. Table 1. The new equations derived in 6 steps Elimination of r 1 2 V,T,p,r :6 V,T,p :5 Vertical coordinate transformation: z to z (unspecified) 3 V,T,p, ,z :7

14. Vertical coordinate transformation: z to z (specified) 4 V,T,p, ,m,f,p :9

15. Going to model thermodynamic variables T’,f’,q,s, 5 V,T,q, ,m,f :8

16. Discretizing in the vertical 6 { { { { { 8 equations; 8 variables; 2 diagnostic

17. Table 2. The new equations coded in 5 steps -1. Introduction of Charney-Phillips grid (staggering). -2. Logarithmic differencing in the hydrostatic equation. From DZ to DlnZ. -3. Incomplete coordinate transformation. From Z to lnZ. -4. Complete coordinate transformation. From lnZ to z . Vertical motion -5. A modified definition of hydrostatic pressure:

18. Step 1 Step 4

19. Step 4 Step 5

20. Definitions & synonyms • Terms that apply to the current model : • Regular grid; • PSEUDO. • For the Development model : • Staggered grid; • Staggered; • STG. • CP

21. Step 1 (staggering)MesoGlobal 42 winter cases

22. Step 2 (+log hyd. eq.)MesoGlobal 42 winter cases

23. Srep 4 (all in log)Step 2 vs 4

24. Step 5 (pi, continuity) 1 case, step 5 validates with 4

25. Step 5 Hydro vs non-hydro (Mesoglobal)

26. Highlights of results obtained in preliminary test runs

27. Temperature at model Lid • Users have often reported noise problems with several variables. This is particularly true with temperature at the model lid. • Problem corrected by the vertical staggering (step1 ). • Possible reduction/removal of sponge on T

28. No sponge at lid : P_PBL_SPNG=0., CSTV_UVDF_8=0.0, CSTV_PHIDF_8=0.0 Weak horizontal diffusion : HZD_TYPE_S=HO, HZD_PWR=6, HZD_LNR=0.04 PSEUDO 28 level Global STG

29. Noise at Tropical Tropopause • Users have often reported noisiness near the tropical tropopause • Problem corrected by the vertical staggering (step1 ). • Possibility of reducing vertical resolution near the tropopause.

30. 28 lev.

31. PSEUDO STG

32. PSEUDO STG

33. Two Delta Z noise • A 2DZ temperature decoupling has often been observed near the surface. • Greatly reduced by the vertical staggering (step 1)

34. LAM example, no physics and no diffusion Temperature profiles near the surface STG PSEUDO

35. Same as previous slide (LAM) • With pseudo, zigzags amplify. • With staggering, zigzags propagate and disappear STG PSEUDO

36. MEC (dernier rapport d’étape)Automne 2007 Par Michel Desgagné, Stéphane Chamberland, Ron McTaggart-Cowan, Michel Valin et Yves Chartier

37. MEC Overview (Modèle Environnemental Couplé) • The objective of the MEC project is to design and develop the next-generation environmental modelling system at RPN • Primary design considerations: • Flexibility – component models must be “plug-and-play” compatible • Modularity – components must be able to operate as independent units • Extensibility – the MEC system must be able to respond to future unforseen requirements

38. MEC Conceptual Model • MEC sequencer controls time stepping, component ordering, distributed parallelism and some basic data flow • “Components” are identified by green boxes called by the sequencer • The GMM (and Whiteboard – not shown) subsystems are responsible for handling data flow requests once the model has been initialized

39. Component API Layer • Each component provides a set of Application Programming Interface (API) “entry points” prescribed by MEC • These APIs have a simple calling sequence, and serve to abstract the component functionality from the sequencer: • Initialization API • Timestepping API • Services API • The APIs are different for each component, but the same for each pluggable module

40. Component API Layer • The sequencer will only communicate with components via the initialization or timestepping APIs • When necessary, components may communicate directly with each other via the services API • The team is also considering using callbacks rather than services APIs – functionality is similar • The use of strict APIs is necessary to enforce the modularity of all components

41. GMM Subsystem • The General Memory Manager is a generic subsystem that will control access to large data structures (e.g. model fields from 1 to 4 dimensions) in MEC • GMM stores data in a memory space allocated on demand along with related metadata, and returns pointers to this space following a get() operation • GMM is fully checkpointable, therefore capable of independently handling the restart procedure for all GMM variables • A prototype GMM subsystem has been created, but a full set of design requirements has not yet been generated

42. Whiteboard Subsystem • Early in the development stage of the top-down simplified model, the need for a storage structure for scalar and one dimensional data became apparent • The Whiteboard acts as a minimalist storage class in its MEC implementation, allowing important limitations to be placed on variable scopes that are not available using standard strategies such as public variables and common blocks • Like GMM, the Whiteboard is fully checkpointable, allowing the subsystem to assist with restarts • A prototype Whiteboard has been developed, and is implemented in the current simplified MEC model

43. MEC Timeline Active Development Previous Development Prototype Implemented Design / Planning Whiteboard Subsystem • Two simplified development models are under active development: • bottom-up – GEM dynamics are connected to a minimalist sequencer, but modularization is incomplete • top-down – highly simplified components are connected to a realistic sequencer, but much of the complexity is absent • Projected dates: • spring 2008: design of I/O components complete and subsystem prototypes implemented • summer 2008: modularized, functional GEM dynamics attached to a realistic sequencer GMM Subsystem Top-down MEC Prototype Functional Prototype Bottom-up MEC Prototype I/O Component Project Documentation Sept April December 2007 2008

44. Additional MEC Information(Environment Canada Internal sites) • Extensive documentation of discussions within the team are available online, as are preliminary documents describing the component requirement and subsystem attributes: http://mrbdoc/doc/dev/mec/mec • A complete set of development tools and simplified models is available from the project subversion repository at: svn://mrbsvn/modeles/mec • The MEC discussion blog is available as a medium for exchanging design concepts and documents the development process – it is available for all contributors at: http://mrbdoc/doc/dev/mec/blog

45. www.ec.gc.ca Lunch Ouranos 8 juillet 2008 Conclusion • CRCM5 (v1.0) = GEMCLIM (v_4.0.x) • Vstag disponible dans GEMCLIM cet automne • MEC premier prototypefonctionnel en début 2009 Merci bien