Collaborators Mark Taylor (Los Alamos) Steve Thomas (NCAR) Rich Loft (NCAR)

1. Multi-Resolution Climate ModelingPrinciple InvestigatorsF. Baer (UMCP)J. Tribbia(NCAR)A. Fournier (UMCP) Collaborators Mark Taylor (Los Alamos) Steve Thomas (NCAR) Rich Loft (NCAR) M. Fox-Rabinovitz (Fac. Assoc.) CCPP Science Team Meeting, San Diego, CA 10/2/01

2. Goal of the Project To develop a climate model methodology which allows for seamless concurrent integration of the planetary scales together with regional scales; To optimize the capabilities of currently available computer hardware (parallel processors) with this model; To incorporate the best features of current state-of the-art climate models.

3. Rationale Variable resolution in a model helps define climate; Nonlinear effects introduced by regional scales must be incorporated into a climate; Smaller scale effects often grow on shorter time scales; Identification and prediction of regional climate should help in understanding the evolution of the global climate. Integrations must be sped up to perform all computations needed for solving the climate modeling problem.

4. The Method and Model SEAM: Spectral Element Atmospheric Model • A global model offering great flexibility and advantages in: • Using geometric properties of finite element methods; • Incorporating local mesh refinement and regional detail; • Utilization of parallel processing; • Maintaining the accuracy of spectral models; • Computational efficiency; • Having no pole problem.

5. Plans for the Model Use as the dynamical core with a forced model like the CCM4; Consider the finite-volume technique; Use semi-implicit integration (currently under study); Consider semi-Lagrange integration; Create an option for linking to an OGCM.

6. Model Domain • Tile spherical surface with arbitrary number and size of rectangular elements; • Inscribe a polyhedron with rectangular faces inside sphere, • Map surface of polyhedron to surface of sphere with a gnomonic projection, • Use the cube (most elementary polyhedron), • Subdivide each of the six faces of the cube as desired. • Can use Local Mesh Refinement (LMR) as desired.

7. Subdivision 1 Cube Uniform Resolution Rectangles

8. An example of Local Mesh-refinement on the globe.

9. Application of the method Use the shallow water equations as an example: • Generate an Integral form of the equations; • Multiply by a global test function () and integrate over the entire domain (spherical surface).

10. spectral element discretization Within each rectangle, use Legendre cardinal functions as basis functions ( ) in each direction.

11. spectral element discretization • The Gauss- Lobatto quadrature points distribute as follows on the local rectangles: • Within each rectangle, estimate the integral equations by Gauss- Lobatto quadrature .

12. Global Test Functions • Simple combinations of Legendre cardinal functions; • One global test function for each grid point. Element interior points Element boundary points

13. Example of the computational system The system of integral equations, Reduces to: A set for element interior points: And a set for element boundary points: where J is the area integration weight.

14. In Summary Tile the sphere with rectangles of arbitrary number and size; Represent the prediction equations in integral form; Use Gauss-Lobatto quadrature for integration; Use Legendre cardinal functions for the basis functions; Use test functions based on the Legendre cardinal functions. These choices result in an extremely simple finite element method with a diagonal mass matrix.

15. Does the model work? • Tests with the shallow water equations: • With test suite (Williamson et al., 1992); • Efficiency; • With local mesh refinement; • With semi-implicit integration; • With high resolution turbulence problem; • With vortex studies. • Tests with a 3-D PE dynamical core and Held-Suarez forcing.

16. Shallow water test case 7 • ICs taken from 500 mb observed data; • Compare spectral element model to NCAR spectral, CSU twisted icosohedral grid, and A-L grid point models. • L2 is normalized error between computed and true height field at end of run.

17. Shallow water test case 7 • Efficiency • Curves show increasing number of elements (M) for fixed spectral degree (N); • Increasing N is more costly than increasing M; • N = 8 appears to be the most efficient index. • Select M for desired resolution.

18. 488x8x8 Mesh Refine 1784x8x8 Global 6936x8x8 • Test case 5 • Ht. field 150x8x8 210x8x8 Local 310x8x8

19. Mesh Refine Zoom 488x8x8 1784x8x8 Global 6936x8x8 150x8x8 210x8x8 Local 310x8x8

20. Shallow water test case 6 • Errors comparing the explicit and semi-implicit integrations of SEAM. Semi-implicit (Thomas and Loft) Explicit Linf (Taylor)

21. Shallow water test case 2 • Ratio of semi-implicit to explicit integrations of SEAM with the shallow water equations for various truncations: • C refers to number of elements; • N refers to spectral degree. • Note that semi-implicit is somewhat faster as well as more accurate. Thomas and Loft

22. Shallow water equations / Jupiter A study of decaying turbulence with high resolution using SEAM; Equivalent depth on Jupiter = 20000 m; Use very weak dissipation; T170, T360, T533, T1033 runs on a CRAY T3E with 128 processors; T1033 has 60000 elements (8x8) ~ 3000 equatorial pts.; Potential vorticity at 276 Jupiter days, T1033.

23. Shallow water equations / Jupiter • Zonal wind at 700 Jovian days (left column); • Equatorial jet strength in time (right column). • Both demonstrate the effect of resolution. Zonal Wind Equatorial Jet Strength 0 100 T170 200 0 100 T1033 200 0 1000 500 Jovian days Latitude

24. Dynamical Core/SEAM/LMR Local mesh-refine Andes topography Global Zoom

25. Dynamical Core/SEAM • Held-Suarez forcing; 384x8x8 ~ T85/L20 SEAM; • Lat. vs Ht. zonal mean wind and eddy variance of T; • Compare T63, G72 and SEAM. U (m/s) T*2 (K2) T63 G72 Latitude Latitude SEAM

26. Dynamical Core/SEAM • Held-Suarez forcing; • SEAM with uniform grid; • Scaling results for various resolutions; almost insensitive to processor number change. HP Exemplar SPP2000 320km/L20 160km/L20 80km/L20 320km/L20 (dotted) 160km/L20 (solid) 80km/L20 (dashed)

27. Dynamical Core/SEAM • Parallel scaling on various computers • Triangles denote SEAM • Horizontal resolution-T181, (g)seaborg- T533 • Other symbols for other models • Log-log plot, flops vs processors. Mflops per processor Gflops # of processors

28. The Research Plan • First and foremost, incorporate SEAM as a dynamical core into CCM4. • Experiments to be undertaken concurrently with this development: • Apply the H-S forced version of SEAM (already running) to meaningful simulations such as turbulence decay; • Develop algorithmic refinements of the 3-D simple forced model to include semi-implicit integration. This is currently under study by our collaborators Thomas and Loft and a test version is running.

29. Research Plan, cont. Mesh refinement studies: Test effectiveness of LMR using 3-D PE version of SEAM with H-S test case forcing model including real topography; Test a suite of long integrations at various uniform resolutions; Test high resolution over the Andes and Himalayas with lower uniform resolution elsewhere; Undertake comparison studies of LMR with the stretched grid model of Fox-Rabinovitz (already begun); Undertake the same comparison with a global model which uses an imbedded fine-scale model (say MM5) if available.

30. Research Plan, cont. Vertical representation: Currently we use -coordinates; we will consider the use of the finite volume method together with spectral elements in the horizontal. Time extrapolation: The model currently uses explicit integration; We will study the feasibility of using semi-Lagrange integration; The semi-implicit scheme is currently under study; it appears to be between 2 to 3 times faster than the explicit scheme for the SWE. Time filtering; We have developed a time filtering scheme which we will test with SEAM. Storm tracks with LMR: We will put LMR into SEAM in the vicinity of quasi-stationary storm tracks to test sensitivity.

31. Research Plan, cont. Studies with SEAM and full physics (CCM4) Using this model, repeat tests described above with the simple forcing model and compare to those results; Space resolution: Test quality of predictions against the standard CCM4 with various resolutions on a uniform grid. Apply LMR: One region at a time; Multiple regions; Consider LMR use in tropics for tropical wave evolution; Compare LMR integrations with other models using LMR, such as Fox-Rabinovitz’ stretched grid.

32. Research Plan, cont. Time resolution: We will determine the effect of SEAM methodology on the climate period selected; i.e., seasonal vs. decadal, etc. We will consider this effect using LMR. Computational efficiency: We will attempt to run the various experiments on machines with largest number of processors to determine scaling.

33. Interactions: NCAR CCM4 staff SCD staff Scientists associated with the project Computers NERSC Computers Support staff UMCP Scientists and students associated with the project Stretched-grid development group. Staff: PIs:Baer, Tribbia, Fournier Co-Investigator:Taylor Faculty Affiliate: Fox-Rabinovitz Collaborators:Thomas, Loft

34. The End