CCSM4

1. CCSM4 Tony Craig NCAR Dec 5, 2008

2. Topics • CCSM4 Overview • CCSM4 Usage • Adding new components • Status of CCSM4 Development

3. Subversion • setenv SVNREPO https://svn-ccsm-models.cgd.ucar.edu • setenv SVNDATA https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ • You can browse these on the web • svn co $SVNREPO/ccsm4/tags/ccsm4_0_alpha38 • http://bb.cgd.ucar.edu/ • Look for “Subversion Issues”, “Introduction to Subversion” • Google subversion

4. ccsm4_0_alpha38 ChangeLog models scripts atm lnd drv ocn ice utils csm_share seq_mct pop xocn socn docn7 dom eshr dshr shr driver pilgrim timing mct pio esmf_tm seq_ccsm_drv.F90 CCSM4 Directory Structure

5. Inputdata • for common shared input datasets that are likely to be reused; grids, external forcing, initial files, etc • local area is /wrkdir/maslowsk/RACM/inputdata subject to scrubbing • permanent area is /archive/u2/wes/maslowsk/RACM • Using these areas • directories should be rwxrwxr-x • files should be r--r--r-- • use unique filenames (date stamped) • never modify or remove ANY inputdata files • some automated scripts for backup and recovery from Gabriele

6. Model Machine Layout(processor layout set in env_mach_pes) Driver lnd ice time ocn cpl atm processors

7. Coupling Issues • Grids • global vs regional • consistent masks/fractions • staggering/averaging • Frequency • tight coupling of atm/lnd/ice (hourly+) • loose coupling of ocn (daily) • Consistency of coupling frequency, model timestep • Conservation, flux accumulation and averaging • Fields • Flux calculations • Albedo/shortwave consistency • Runoff

8. Topics • CCSM4 Overview • CCSM4 Usage • Adding new components • Status of CCSM4 Development

9. Compsets, Grids, Machines • Note: Compsets and Grids have a short name and a long name • Compsets are the combination of components used • X = all dead models • A = all data models • B = all active models • others are C, D, F, G, H, I • Grids define the resolution • standard resolutions have lnd grid = atm grid, ice grid = ocean grid (f19_g15) • recently added “trigrid” support, atm and land grids different (f45T42_g35) • Machine is, for example, midnight > cd scripts > ./create_newcase -list

10. create_newcase (1) > cd scripts > cat README > ./create_newcase -help • must specify case name, compset, resolution, and machine. these cannot be changed inside the case directory. • always use -skip_rundb option > ./create_newcase -case Btest -res f45_g35 -compset B -mach midnight -skip_rundb • a case called Btest will be generated in the scripts directory • the -case argument can be dir/name

11. create_newcase (2) > cd Btest • edit env files • env files are mixture of xml and csh • env_mach_pes sets the pe layout. this must be done before configure is run. • env_run sets run length, restart frequency, etc • set LOGDIR to $CASEROOT/logs in env_mach_generic • set DOUT_S to FALSE in env_run.xml > configure -case • configure generates build and run scripts plus the Buildnml_Prestage and Buildexe directories • some env files are locked after configure. to reset, type “configure -cleanall”

12. create_newcase (3) > ./Btest.midnight.build > qsub Btest.midnight.run • The case will build under $EXEROOT, $EXEROOT is set in env_mach_generic.xml • The case will run under $EXEROOT/ccsm_se • Things I haven’t described: • setting up a full production run • setting the pe layout • archiving • how the scripts work • Buildnml_Prestage and Buildexe directories

13. Output files • LID is a date string, yymmdd-hhmmss created by CCSM4 everytime it runs to generate unique filenames • log files, *.log.$LID • each component has own log file. stdout/stderr are redirected to ccsm.log.$LID. • look in $EXEROOT/ccsm_se • look in $LOGDIR if set • look in short term archive area if DOUT_S is TRUE • timing files • directory in $CASEROOT/timing • ccsm_timing_summary.$LID has performance summary • history and restart files • look in $EXEROOT/ccsm_se • look in short term archive area if DOUT_S is TRUE

14. create_test (1) > cd scripts > ./create_test -help • must specify testname; $test.$resolution.$compset.$machine • recommend using -testid > ./create_test -testname ERS.f45_g35.B.midnight -testid t001 • create_test automatically configure the test, designed to require no user intervention. • will create a test named ERS.f45_g35.B.midnight.t001 • -testroot option allows the testcase to be place in another directory • generate and compare are supported for regression testing

15. create_test (2) > cd ERS.f45_g35.B.midnight.t001 > ./ERS.f45_g35.B.midnight.t001.build > qsub ERS.f45_g35.B.midnight.t001.test • Test results are in the file TestStatus • The file TestStatus.out provide more test result details • log files should be copied into the test directory under a directory called logs

16. Topics • CCSM4 Overview • CCSM4 Usage • Adding new components • Status of CCSM4 Development

17. *_comp_mct.F90 (1) • The *_comp_mct.F90 provides the coupling between the driver and the component. This is the top level of any component • Driver looks like call lnd_init_mct() while (.not. stop_alarm) call seq_timemgr_clockAdvance() call lnd_run_mct() end do call lnd_final_mct() • Must have an init, run, and finalize method

18. *_comp_mct.F90 (2) • Interface generally contains • ESMF_Clock • seq_cdata datatype that holds • mct_gGrid (domain information) • mct_gsMap (decomposition information) • seq_infodata_type (scalar data) • MPI Comm, Component ID, etc • Attribute vectors (2d fields), one for data to the component, one for data from the component. • Use an existing version and modify it for your component

19. init_mct method • enter • extract mpicom using seq_cdata_setptrs. this is THE mpi communicator for your component. • get ccsm4 settings using seq_infodata_GetData() • initialize and synchronize your component clock using seq_timemgr_EClockGetData() • set the global seg map which describes the grid decomposition of your component. • set the global grid which prescribes grid field data of your component. • allocate the attribute vectors to be used to pass data back and forth. • pack the export attribute vector with initial data • set some ccsm4 settings using seq_infodata_PutData() • return • NOTE: there are literally dozens of examples as each component has implemented this. Review and Reuse that code.

20. run_mct method • enter • check ccsm4 settings using seq_infodata_GetData() • get clock information using seq_timemgr_EClockGetData() • unpack the import attribute vector • “run your component” • pack the export attribute vector • set some ccsm4 settings using seq_infodata_PutData() • verify the internal clock is in sync with the CCSM4 clock • return

21. Coupling Interface Final Notes • The driver will hand you an mpi communicator. You must use this inside your component. • The components must check that they are staying in sync with the driver clock. • The component must respond to a restart flag sent from the driver. • The albedo/shortwave interaction between components is complicated and critical. This will probably cause you problems at some point.

22. A few other issues • Unit numbers • History files • Namelist input • Mapping files for coupler

23. Component Template Scripts • Under each component is a bld directory and a script called *.cpl7.template. • This script is what the CCSM4 scripts execute to generate component specific scripts under your $CASEROOT Buildnml_Prestage and Buildexe directories. • New components will have to implement a version of this for their component.

24. Topics • CCSM4 Overview • CCSM4 Usage • Adding new components • Status of CCSM4 Development

25. Status of CCSM4 development • beta series • New science • GLC model • scripts continue to evolve • www.cgd.ucar.edu/cseg