Lessons learned from building and managing the Community Climate System Model

1. Lessons learned from building and managing the Community Climate System Model David Bailey PCWG liaison (NCAR) Marika Holland PCWG co-chair (NCAR) Elizabeth Hunke PCWG co-chair (LANL) David Lawrence LMWG co-chair (NCAR) Steve Vavrus University collaborator (U. Wisconsin)

2. A brief history • 1993 – Initial meetings of NCAR scientists • 1994 – Development of the first coupled model • 1996 - First successful coupled simulation (little drift); first Breckenridge workshop; Working groups formed; first release to the research community • 2000 – name changed to Community Climate System Model • 2004 – CCSM3 released • 2009 – CCSM4?

3. CCSM Management CCSM Advisory Board (CAB) Scientific Steering Committee (SSC) Chair: Peter Gent An advisory committee consisting of university faculty, members of national laboratories Provide scientific leadership; oversight of activities of working groups, coordination of model experiments, decision making on model definition and development priorities Working Groups: Development and Applications CCSM Sponsored NSF and DOE

4. Chemistry Climate BioGeo Chemistry Atm Model Land Model Polar Climate Ocean Model Climate Change PaleoClimate Climate Variability CCSM Working Groups Development Application Software Engineering CCSM is primarily sponsored by the National Science Foundation and the Department of Energy

5. Makeup of CCSM Working groups • 2 or 3 co-chairs, at least one external • Scientific liaison • Software engineer (ideally one per WG) • ~10 or more hands-on model developers • >40 total participants, majority are external • Participation is voluntary, long-term participation is common

6. Developing and maintaining community participation • Meetings (webcast when possible) • Plenary (13th annual CCSM workshop, June 17-19, Breckenridge) • Component • Frequent local CCSM scientist • Encourages community involvement • Setting priorities, coordination of activities, putting names to tasks

7. Lessons: Resources • Local dedicated computing and software engineering resources • Sufficient staff with base funding • Scientific model developers and users with considerable local in-house scientific expertise and scientific investment throughout project • Software engineers (6 people minimum component liaison, coupling, run scripts, testing) • Supplemental grants for specific projects

8. Lessons: Management of model • Well-documented, user-friendly, tested, and efficient model • Policies and procedures for ownership and distribution of model and model data • Process for integrating community model improvements • Policies for community support / external user problem solving

9. Lessons: Model development • Started small, built up from models existing at NCAR • Tuning a coupled model is a slow process (6 months or more for CCSM3) • Deadlines, such as release dates or IPCC, keep progress moving along • Conflicts will happen (SSC, Working group co-chairs) • Compromises may be required • Near-surface ocean eddy flux change killed Atlantic layer • Snow cover fraction • Sea ice/snow albedo tuning • Applications community needs to be involved

10. CCSM3.5 CCSM3 OBS Not all good news in CCSM3.5 - Arctic Ocean Profiles Distinct Atlantic layer missing in CCSM3.5 Runs. Does not appear to be related to ice model changes Considerable cooling of waters at depth compared to CCSM3 Salinity profiles still look quite good.

11. Lessons: Model development • Starting small, built up from models existing at NCAR • Tuning a coupled model is a slow process (6 months or more for CCSM3) • Importance of setting deadlines (release dates) • Version control (e.g. Subversion) • Conflicts will happen (SSC, Working group co-chairs) • Compromises may be required • Near-surface ocean eddy flux change killed Atlantic layer • Snow cover fraction • Sea ice/snow albedo tuning • Applications community needs to be involved

12. Lessons: Coupling is hard • High quality component models does not ensure high quality coupled simulations • Tuning the coupled system is not easy • Difficult to define and agree on metrics • Examples of unforeseen problems • CLM3.5 crashed ocean model in Arctic (runoff spikes) • Smagorinsky parameter caused sea ice bias only in FV Runoff Runoff clim

13. Lessons: Coupling is hardCase of the wayward oceanic Smagorinsky parameter Spectral core FV core

14. Lessons: Software development • Version control (e.g. Subversion) • Software gatekeeper for each component • Permits new model development to keep up and not conflict with accepted model code • Regression testing • Flexible build/run system • User-friendly • Scripts • Code • Diagnostic packages • Validation packages

16. Summary • Lessons learned from CCSM experience • Building and maintaining a vibrant model- development and user community is critical • Requires availability of sufficient resources from computing to software engineering • Coupling is hard and compromises may sometimes be required • Deadlines help, driving force • User-friendly is key

17. Timeline of Climate Model Development