CCSM Software Engineering Update

1. CCSM Software Engineering Update Tony Craig CCSM SEWG Meeting Feb 4, 2003

2. Outline • CCSM SE Organization / CSEG Staff • CCSM Release Status • Process Update • ESMF • SciDAC • CPL6 Update • CLM2.1 Update • New Projects: • GUI and DataMover • Document control • Data models rewrite • Testing status • Vectorization

3. CCSM Entities • SSC = Scientific Steering Committee • WG = Working Groups (about 8 total) • SEWG = Software Engineering Working Group • CSEG = CCSM Software Engineering Group • ESMF = NASA funded coupling framework project • SciDAC = DOE funded project to improve CCSM

4. SSC WG WG WG WG WG WG WG SCIDAC ESMF SEWG WG CCSM Organization(SE perspective) CSEG

5. CSEG Staff Changes • 9 Software engineers • 2 new hires, Spring, 2002 • Wei Yu - performance and testing • Dan Miner - infrastructure • 1 reclassification, Spring 2002 • Erik Kluzek - SEIII (ESMF) • Lawrence Buja moved to CCR, Fall 2002 • 1 new hire, Dec 2002 • Tom Henderson - SEIII • 1 open position (SEII/III)

6. CSEG staff • Tony Craig (CCSM integration) • Nancy Norton (ocean model, data models) • Tom Henderson (atmosphere model) • Brian Kauffman (coupler model) • Erik Kluzek (ESMF, atmosphere model) • Dan Miner (infrastructure) • Julie Schramm (ice model, doc project) • Mariana Vertenstein (land model, GUI) • Wei Yu (testing, GUI)

7. CCSM2.0 Release Status • CCSM2.0 released May 17, 2002 • Source code • Testing • Documentation • Input datasets, control run output (b20.007) • Web page development • Patch 1, 2, 3, 4 • Tutorial

8. CCSM2.0+ Release Status • CCSM2.0.1 release scheduled mid-July, 2002 Oct 2002 • Bit-for-bit with CCSM2.0 for b20.007 control • Includes paleo version • Includes updated datm and dlnd components and forcing datasets • Other minor SE enhancements • Patches • CCSM2.1 release expected Fall, 2002 • Will improve the scientific results • Other SE enhancements

9. CCSM2.1+ Release Status • CCSM2.1beta release expected March 2003 • “beta” release - no control, no long-term support • CPL6 • CLM2.1 • Minor physics changes • CCSM2.2(?) release expected Fall 2003 • IPCC version • full support

10. Process Update • Release policy, support (2+1 years, patches) • CCSM CRB • CAM CRB • CVS Repository access • Infrastructure • New CVS server, Update CVS • New bug tracking tool • Database (experiments, bug tracking) • Integration of CVS, bug-tracking, GUI, database • Testing Brown Bag Discussions

11. ESMF • Continue to use ESMF pre-release library in CCSM • Close working relationship with ESMF Core Team • Design and prototyping • Writing pseudo-code • ESMF will not be integrated into the production system until usability, robustness, and performance are clearly demonstrated • Erik Kluzek leading ESMF effort in CCSM

12. SciDAC • CAM improvements • CAM/CLM coupling • CLM runoff model • CPL6 • POP2 • CSIM/CICE updates • Cray X1

13. CCSM GUI Project Goals • Greater flexibility in building and running CCSM on a variety of platforms, machines, and configurations; will set better default task and thread counts; will handle batch queues better • Increased error checking • Capability to edit component namelists • Will provide capability of starting from previous experiments (including modified source code) and modifying those experiments or creating new experiments

14. CCSM GUI Project Goals (2) • Capability to build, pre-stage input data, or run the CCSM independently • Written in java (can run on a much larger number of platforms). • New DataMover tool • Functionality to move a file, a list of files or a list of directories from one location to another by a variety of methods that may be user-prescribed • Very Flexible • Will support recursive directory movement

15. CCSM GUI Project Status • Requirements document exists • Several month process gathering and refining requirements • Prototyping and Design underway • Regular internal reviews • Mariana Vertenstein leading effort

16. Document Project Overview • Motivation: • As more documentation is being written for CCSM, documents need to become better organized • CCSM Documents include: • Version specific documents (Users Guides, …) • Component general documents (Requirements Doc, …) • CCSM general documents (Developers Guide, Test Doc, …) • Document templates • Goals of project: • Provide a central location for all CCSM documentation • Get all CCSM documentation under revision control • Create an online library where all CCSM documents will be accessible

17. Document Project Status • There are requirements for the document organization • Five proposals have been presented, based on these requirements. • Two proposals would use current CCSM CVS module • Easiest to implement • Constrained by directory structure of current repository • Three proposals create a separate CVS module for documentation • Source code and document tagging more complex • New directory hierarchy could be a desirable option • Julie Schramm leading effort

18. Data Model Project Background • The CCSM Data Models were originally created to serve as simple "stand-ins" for their active-model counterparts. • They shared common functionalities, structure, and coding style. • Individual data models have been modified by developers and scientific groups for science capability, SE left behind. • Current data models lack consistency and are difficult to upgrade, use, and maintain. • Why update models now? • Scientist frustrations • Platform for ESMF prototyping • Take advantage of cpl6 share code • Opportunity to develop new share code

19. Data Model Project Status • Gathering and recording requirements for data models • Review status of each requirement. • Solicit input from the scientific groups who maintain specialized data model code. • Create a design plan for the data models, isolating as much as possible those elements common to all data models. Develop a library of shared code. • Nancy Norton leading effort

20. CPL6 update • CPL5 is still the production coupler • Active and data components can couple with CPL6 • New dead component available for testing • Completing validation comparison tests with CPL5 • Will begin longer production run soon • Assessing performance • Same science • Expected to be the CCSM production coupler in March, 2003

21. Mapping: ocn -> atm Ocn (122,880 points -> Atm (8192 points) bundle of 9 fields 120 mapping calls

22. CLM2.1 update • Complete rewrite of CLM2.0 • introduction of new data structures • introduction of new history output mechanism • Improved flexibility for incorporation of new land model physics (e.g. biogeochemistry) • Upgrade to CAM/CLM coupling • Same performance in CCSM • Same science • CLM2.1/CAM2.0.2 release expected soon

23. Testing Status • Weekly testing of CCSM2.0, CCSM2.0.1 releases on chinook, blackforest, bluesky, seaborg • Automatic testing of CAM every night after a commit on the primary development branch • Formal Test Plan Document under development

24. Vectorization update • “Vectorization is NOT a high priority” (SSC) • Earth simulator still primary target for vectorization • CAM/CICE rewrites happening in community • CLM rewrite will start after CLM2.1 release, community effort • CPL rework will start after CCSM2.1 release (cpl6), community effort • New opportunity on CRAY X1, DOE taking lead

25. Summary • IPCC release next major target, Summer 2003. • New projects • CCSM GUI and DataMover • Documents organization • Data models rewrite • CPL6 and CLM2.1 “complete” • Better testing • Continue to improve process