The SciDAC2 CCSM Consortium Project 2007 - 2012

1. The SciDAC2 CCSM Consortium Project2007 - 2012 Presented by John Drake Atmospheric Model Working Group Meeting January 31, 2007, NCAR, Boulder

2. Lead PI: John B. Drake, Oak Ridge National Laboratory, Co-Lead PI: Phil Jones, Los Alamos National Laboratory Argonne National Laboratory (ANL) Robert Jacob, Ray Loy, Brookhaven National Laboratory (BNL) Robert McGraw; Lawrence Berkeley National Laboratory (LBNL) Michael Wehner,Lawrence Livermore National Laboratory (LLNL) Phillip Cameron-Smith, Arthur Mirin; Los Alamos National Laboratory (LANL) Scott Elliot, Philip Jones, William Lipscomb, Mat Maltrud; National Center for Atmospheric Research (NCAR) Peter Gent, William Collins, Tony Craig, Jean-Francois Lamarque, Mariana Vertenstein, Warren Washington; Oak Ridge National Laboratory (ORNL) Marcia Branstetter, John Drake, David Erickson, Forrest Hoffman,W. M. Post, Trey White, Patrick Worley; Pacific Northwest National Laboratory (PNNL) Steven Ghan, Xiaohong Liu, Richard Easter, Rahul Zaveri, Sandia National Laboratories (SNL) Mark Taylor Goals and Deliverables Over the next five years, we will pursue this objective through four integrated goals: Extend the capabilities of the Community Climate System Model (CCSM) to include representations of biological, ecological, chemical, and aerosol processes that will allow scientists and policy-makers to simulate climate and climate change using a comprehensive Earth system model, Provide the necessary software and modeling expertise to rapidly integrate new methods and model improvements, Pursue the development and evaluation of innovative methods in the coupled context of the CCSM, and Improve the performance, portability and scalability of the CCSM on available and future computing architectures for use in national and international assessments of climate change. Current Activities (cont.) Scientific Challenge To determine the range of possible climate changes over the 21st century and beyond through simulations using a more accurate climate system model that includes the full range of human and natural climate feedbacks with increased realism and spatial resolution Objective The primary objective of this proposal is to develop, test, and exploit a first generation of Earth system models based upon the CCSM. We will bring to the community a new well-validated version of CCSM that will run efficiently on thousands of processors and include significant model enhancements. Improvements to the representation of carbon and chemical processes for treatment of greenhouse gas emissions and aerosol feedbacks will be performed in collaboration with the DOE Atmospheric Science Program, DOE Atmospheric Radiation Measurement Program, and DOE Terrestrial Carbon Program. This project will enable climate change simulations required for scheduled national and international climate change assessments to which the CCSM is committed as part of the National Climate Change Science Program (CCSP) strategy. Biogeochemical Coupling: • Carbon Land Model Intercomparison (C-LAMP): • Determine the best set of terrestrial biogeochemistry process representations to include a future release of CCSM • Three models coupled to CCSM3: CLM3-CASA', CLM3-CN, and LSX-IBIS • Experiments are being performed with changing climate, CO2, nitrogen, and land use • Comparison with high frequency, high quality flux tower measurements The SciDAC2 CCSM Consortium Project:A Scalable and Extensible Earth System Model Atmospheric CO2 resulting from ecosystem respiration from the CLM3-CASA’ Aerosols and Atmospheric Chemistry • Aerosols alter cloud properties and the distribution of radiation in the atmosphere • Development of a faster chemistry package for the troposphere and lower stratosphere • Introduction of modal and moment representations of aerosol dynamics modeling aerosol size distribution • Modeling the surface emissions of pollutants changes atmospheric radiation High Resolution Ocean and Cryosphere: Tasks and Organization Earth System Model • Terrestrial carbon cycle and dynamic vegetation • Atmospheric chemistry and aerosol dynamics • Ocean ecosystems and biogeochemical coupling • Feedbacks between atmospheric composition and biogenic emissions Model Integration and Evaluation • Integration and unit testing • New icesheet and ocean models • New atmospheric dynamics: finite volume (cubed sphere), discontinuous Galerkin, others(icosahedral) • Frameworks for model evaluation Computational Performance • Scalablity to thousands of processors, load balance, (fault recovery) • Ice sheet model: • Largest missing piece of physical climate model and necessary for sea level rise prediction • Variable resolution for melting at margins of ice sheet • Ocean vertical grids: • Arbitrary Lagrangian-Eulerian vertical grid for deep ocean circulation with good resolution in the surface mixed layer • Ocean horizontal grids: • Eddy-resolving simulations required for adequate simulation of ocean circulation • Unstructured grids provide ability to focus grids in eddy-active regions at reduced cost Collaborations Scientific Applications (SAs) and Partnerships(SAPs) Brookhaven National Laboratory Robert McGraw Oak Ridge National Laboratory Patrick Worley Argonne National Laboratory Kotamarthi Rao Centers for Enabling Technology Collaborations Earth System Grid (ESG) - Dean Williams Performance Engineering (PER) – Bob Lucas VACET – Wes Bethel ; SDM - Ari Shoshani TOPS – David Keyes; ITAPS - Lori Diachin APDEC - Phil Colella CVT Delaunay tessellation of the North Atlantic with 85K triangles - Gunzburger and Ju Scalability and Performance: • Software enabling scalability: • Collecting performance data to quantify and identify scalability limiters. Modifying code to improve scalability: • • Enabling different degrees of parallelism in physics and dynamics of atmosphere model. • • Identifying and eliminating nonscalable memory allocations. • • Working with Pnetcdf developers and CCSM software engineers to implement a single parallel I/O interface across all CCSM components. • • Validating software solutions on large Cray XT3 and IBM BlueGene systems. Computational Climate End Station for Climate Change Science - Warren Washington (PI) Current Activities Model Integration and Evaluation: • Software enabling integration: • Model Coupling Toolkit (MCT 2.3.0) released (1/10/07) • Optimized coupling strategies • New methods for model evaluation (w/ Rao) Sea surface temperatures simulated on 0.1o displaced pole grid using the POP2 code Contact Info: drakejb@ornl.gov, pwjones@lanl.gov Project Web Page: www.scidac.gov/climate/earth.html

3. AMWG Related Activities • Atm chemistry and aerosol dynamics (J.-F. Lamarque) • Aerosol Indirect effect - Steve Ghan, Bill Collins • Moment methods - Bob McGraw • Atm chem fast mechanisms - Philip Cameron-Smith • Sulfur cycle - Dave Erickson • Model Integration and Evaluation (M. Vertenstein) • Integration and unit testing (Rob Jacobs, Tony Craig) • FV (cubed sphere), DG, others (Mark Taylor, Art Mirin, Pat Worley, J. Drake) • Evaluation Frameworks (K. Rao et al.) • Computational Performance (Pat Worley, Art Mirin, Ray Loy) • Scalability, load balance, (fault recovery)

4. Supercomputers at NCCS • 50 TF Cray XT3 • 5212 dual core Opterons • 21 Terabytes memory • 100TB scratch 1000 TF Cray TBD 250 TF • 60 TF Cray XT4 • 6296 dual core Opterons • New interconnect • Acceptance testing • 18 TF Cray X1 / X1E • Will not expand • 1024 processors • Vector processing for sustained performance 100 TF 50 TF 25 TF 18.5 TF 2009 2004 2005 2006 2007 2008 Leadership Computing Facility (also BG/L at Argonne)

5. Computer Allocation:Computational End Station for Climate Change Science • Extensible community models available for computational science • Coordination of effort among agencies and institutions (talk to Lawrence Buja, J. Drake or one of the partner representatives) PI: Warren Washington (NCAR), partners: CCSM, COSIM, PCMDI, SciDAC, NASA-GSFC, PNNL, CCRI(Universities)

6. What does DOE want? • Relation to Aerosol Science Program and Terrestrial Carbon Program • Coordinated enterprise, relation to CETs and SAPs • Reporting • Impacts and revised scope statement • 30 days: 4-6 slides for Dr. Orbach • 60 days: Management Plan, website, performance baseline • 6months: progress reports, highlights • DOE OBER Performance Targets • 2006 Deliver new measurements of clouds where observations are missing • 2007 Include realistic cloud simulations in a climate model • 2008 Measure ecosystem responses to climate change • 2010 Develop/validate new models predicting effect of aerosols on climate forcing • 2010 Provide climate model that links the Earth climate system with Earth’s biological systems • 2013 Reduce differences between observed temperature & model simulations at sub continental scales using several decades of recent data • 2015 Deliver improved climate data & models for policy makers to determine safe levels of greenhouse gasses.