NICAM : DX=3.5km

1. The 2010 Workshop on the Solution of Partial Differential Equations on theSphereAugust 24-27, 2010 Alteration of vertical grid in NICAMtowards the super-high resolution simulationsHirofumi TOMITARIGC / JAMSTEC NICAM : DX=3.5km MTSAT-1R

2. Outline • Introduction of NICAM • Summary of current version of DC • Vertical problem • Horizontal high resolution run • PGF error • Land surface model coupling • Future research plans using the next generation supercomputer in JAPAN • 10PFLOPS machine in Kobe • Named “K-computer” • Summary

3. Summary of NICAM dynamical core

4. NICAM dynamical core (1) • Horizontal grid • Icosahedralgrid • modification by spring dynamics • Reallocation of grid to gravitational center of control volume Tomita et al. 2001,2002, JCP

5. NICAM dynamical core (2) • Vertical grid • Terrain-following coordinate • with Lorentz grid • Governing equation • Non-hydrostatic equation • with deep atmosphere Tomita & Satoh 2004, Fluid.Dyn.Res Satoh et al.2008 JCP Continuity eq. Horiz. ,mom. eq. Vert. ,mom. eq. Energy. eq.

6. NICAM dynamical core (3) • Solver • Split explicit method • Fast mode ( e.g. acoustic wave, gravity wave ) • Small time step ( forward-backward ) • Slow mode ( e.g. advection term ) • Large time step ( by RK2 or RK3 ) • In slow mode • Horizontal explicit / vertical implicit scheme

7. NICAM dynamical core (3) • Horizontal advection scheme • Miura ( 2007, MWF ) scheme with Thurburn(1996) flux limiter • Simple way of flux estimation • 2nd order and monotonicity. • Consistency With Continuity in tracer advection. • Niwa et al. (2010) • Operator splitting into horizontal and vertical directions • Use of intermediate density (Easter(1993))

8. Option of other grid configuration in NICAM • icosahedral grid • different topology • Non-uniform resolution(Iga 2010, MWR submitted ) • Stretched (Tomita2008)

9. Vertical problem of dynamical core at the high-resolution runs

10. Numerical problem • Owing to steep mountain& terrain following coordinate • In TFC, one reference state should be set in all the region. • Limited area model • No problem • Global model • Difference from reference state is large. • > 50K between tropics and polar region. • Pressure gradient force error! • Remedy : setting the mid-latitude profile as the reference state. • Tibetan Platau, Rocky mountain, Andes mountain are in the mid-latitude. The simulation result depends on the reference state!

11. Sometimes, the model blow up at the high resolution simulation! • PGF error acts violently! • So-called “hydrostatic inconsistency • Reconsideration of vertical discretization from the terrain-following coordinate to height basis coordinate. • Vertical adaptive mesh for the PBL scheme. The problem becomes severe when the resolution increases!

12. Problem about coupling with land model(1) • Energy balance equation at the interface • Usual way: • explicit estimation of bulk coef in the surface flux : using the previous skin temperature.  oscillatory of skin temp. Long wave Sensible heat Ground flux Short wave latent heat Typical diurnal variation of skin temperature At dawn & suset, bulk coef. is oscillatory!

13. Problem about coupling with land model(2) • Best Solution • Atmosphere vertical turbulence & surface energy balance is implicitly solved at the same time. • However,,,,, • Modulization? • We want to pursue the separate modules for atmosphere turbulence and land surface model. • Another solution • The bulk coefficient at the surface is implicitly solved! Tomita (2009, JHM)

14. Research plans toward the next-generation supercomputer

15. The next-generation supercomputer in Kobe • The next-generation supercomputer system (~2012) • System • One nodes: • 128GFLOPS • Memory band width : 64GB/s • B/F : 0.5 • 80k nodes/640k core • Peak performacne : ~10PFLOPS • Total memory : ~1PB • Network 3D torus bi-direction 5GB/s X6

16. What can we expect on it? Courtesy of Prof. H.Tanaka(Tsukuba Univ.) Miura et al. 2007 Next-generation supercomputer strategic program--- Field 3：Research of prediction of weather and climate contributing to disaster prevention • Change of tropical cyclone in the future climate • NICAM3.5km-model (7km model also) • Intensity • Frequency • Local information • Integration time : 10 years order • More reliable statistics • Predictability of tropical weather • NICAM3.5km, 1.7km • MJO, tropical cyclone, monsoon • How long can we extend the tropical prediction by GCRM?

17. Grand challenge! • 野田暁氏提供 Global 400m mesh run • Collaboration project with RIKENand JAMSTEC • Horizontally 400m with 100 vertical levels • Truly, global cloud-resolving! • Use of the full computational resource • 1PBmemory • All computational nodes • Integration time ：1 week? • Purpose： • Improve PBL cloud? • Improve detail structure of cumulus? • Diurnal cycle? • Computationally, • Well work in the case of full use? • Suggestion to the next-next computer systems plan

18. How about the sustained performance? • Vector machine • Problem size:7km/L40 • ES：３０～４０% • ES2:１０～１５% • Scalar machine ( NO TUNING! ) • Cray XT4 (AMD Opteron ) : 2.9%(@ Oct. 2009） • collabolation with COLA & ECMWF • T2K tsukuba(AMD Opteron) : 3.3% • IBM :BlueGene/P : 6% • Good balance between memory access speed and CPU speed • Recent PC : Core i7(nehalem core)6% • Problem size : 240km/L54 • (owing to memory band widith:6.4GB/sX3=19.2GB/s) • B/Fratio：0.45 • The Next Generation Machine(BF : 0.5) • Default : 5% • Medium tuning : 10% • Highly tuining : 20%???

19. Summary • NICAM: • Global non-hydrostatic atmospheric model • Icosahedral grid • Vertical discretization: • plan to change from terrain following coordinate to height based coordinate. • Future plans: • On the next-generation supercomputer system • Change of tropical cyclone in the future climate • Predictability of tropical weather • Grand Challenge! ( 400m global mesh with 100 levels. )