ESMF Update

1. ESMF Update ESMF Joint Exec/Adv Meeting Cecelia DeLuca / NCAR October 16, 2007

2. Outline • Achievement Highlights • Development Plan Through 2010 • Grid Rework • Improving Ease of Use • Organizational Status • View Ahead

3. ESMF Highlights FY07 • Technical • First release of redesigned grid software, July 2007 (ESMF 3.0.3) • Significant reductions in open bugs and support requests • ESMF Interoperability and Integration Initiative (ESMFI3) began in May 2007 to address clearer compliance terms and the role of metadata in interoperability • COAMPS-NCOM coupling using ESMF sparse matrix multiply for regridding passed beta test September, 2007 • Number of components at the working prototype level or beyond increased from 36 at end FY06 to 58 at end FY07 • Organizational • ESMF and related projects were promoted to a Section (Earth System Modeling Infrastructure Section) at NCAR October 2006 • Developed user survey, first deployment will be January 2008 • Established operations manager role • Expanded administrative assistant role and wrote Administrator’s Guide • New project website with more introductory and overview material – values, vision, FAQ, etc.

4. ESMF Highlights FY07 • Programmatic • National Unified Operational Prediction Capability pilot project approved to begin construction of next-generation NWP system based on ESMF • New award from NOAA: NOAA ESMF-based Modeling Architecture, $600K over three years for core team development, PI Steve Lord • Supplementary award from DoD HPCMP for core team development through the Battlespace Environments Institute, $1.5M over three years • Successful completion of Community Climate System Model (CCSM)-ESMF Stage 1 Plan Evaluation showing < 5% overhead to time to solution on a range of platforms, comparable memory use, acceptable build • CCSM-ESMF Stage 2 Evaluation Plan approved by CCSM Scientific Steering Committee.

5. Outline • Achievement Highlights • Development Plan Through 2010 • Grid Rework • Improving Ease of Use • Organizational Status • View Ahead

6. ESMF Release Plan 2002 2003 2004 2005 2006 2007 2008 2009 2010 ESMF v1 Prototype ESMF v2 Components, VM and Utils ESMF_GridCompRun() ESMF v3 Index Space Operations ESMF_ArraySparseMatMul() ESMF v4 Grid Operations ESMF_GridCreate() ESMF_FieldRegrid() ESMFv5 Standardization Build, init, data types, error handling, … Last public ESMF v2.2.2r Last internal ESMF v3.0.3

7. Many Basic Elements Completed … • Established component-based architecture • Serial or parallel execution • Components can run concurrently or sequentially • Single executable support, limited support for multiple executables • Support for model ensembles • Acceptable build, portable code • Utilities such as time manager, message logger, and resource file manager are usable • Reference Manual, Users Guide, ~2500 tests and examples

8. What Defines Done for 2010? • Single- and multi-tile index space representation and operations, including parallel sparse matrix multiply for regridding with user-supplied interpolation weightsComplete except for some aspects of multi-tile connectivity • Full grid representation for curvilinear, single- and multi-tile, and unstructured gridsRectilinear and curvilinear single-tile grids complete, multi-tile not complete, unstructured prototype needs Fortran interfaces, testing, and documentation, all grids need integration with higher level classes • Support for regular, irregular, and arbitrary data distributionsComplete for regular and irregular distributions, arbitrary distributions needed • Parallel regridding package capable of a variety of methods (bilinear, conservative, higher order) for use with any grids that can be representedPrototype complete, needs Fortran interfaces, testing, and documentation • Performance of core functions equal to or better than comparable packages and native model codeTrue for codes tested

9. What Defines Done for 2010?, cont. • Tutorial material that guides users through core functionsSome but better materials needed • Fortran and C interfacesC interfaces needed • Clear compliance termsInitiative to address the issue (ESMFI3) has outlined compliance levels, specific terms needed and automatic verifier desired • High level of consistency in behavior, interfaces, and documentationOngoing • Few bugsOngoing Remaining work is mainly in the areas of grid development and improving ease of use

10. Outline • Achievement Highlights • Development Plan Through 2010 • Grid Rework • Improving Ease of Use • Organizational Status • View Ahead

11. Grid Rework ESMF initially focused on representing and coupling the rectilinear, single-patch grids commonly used in climate applications. The ESMF team is redesigning its datastructures for the representation of data blocksand distributions so that they can accommodate the multi-tile, curvilinear grids and unstructured grids found in ocean, weather, space weather, and hydrological models. Making the transition … from transformations between regular grids and distributions … to transformations between grids that are complex in topology and distribution

12. ESMF Data Structure Redesign • Goal: move from data structures that can only represent single-tile, rectilinear grids to data structures that can represent multi-tile, curvilinear grids and unstructured grids • Design reviews began January 2005 Bundle V3.0.0 May, 2006 Bundle Field V2.2.2 March, 2006 Field Grid Array NewArray Grid Array DistGrid LocalArray NewDistGrid DistGrid Can do: index space representation and operations, regridding via sparse matrix multiply LocalArray DELayout DELayout distribution Bundle V3.0.3 July 2007 Field Public class Private class To be replaced Reworked Wholly new Grid Array NewGrid NewArray DistGrid LocalArray NewDistGrid Can do: rectilinear and curvilinear grid representation DELayout

13. ESMF Data Structure Redesign, cont. Bundle Bundle V3.1.0 October, 2007 Field Field Now Array NewGrid NewGrid NewArray NewArray NewDistGrid LocalArray LocalArray NewDistGrid Can do: representation of gridded fields Can do: representation of gridded fields DELayout DELayout Bundle V3.1.1 January, 2008 Field Public class Private class To be replaced Reworked Wholly new NewGrid NewArray LocalArray NewDistGrid Can do: field and bundle regridding and operations DELayout

14. Final Step Bundle V3.1.* 2008/2009 Field NewGrid NewArray Mesh LocalArray NewDistGrid Can do: both logically rectangular and structured grid representation and operations DELayout Public class Private class To be replaced Reworked Wholly new

15. Analytic Wind Stress:Interpolation UsingPatch Recovery Curl of bilinear interpolant Exact curl Curl of patch-recovered interpolant Work conducted by David Neckels/ESMF in collaboration with Bill Large of the ocean modeling section at NCAR.

16. Outline • Achievement Highlights • Development Plan Through 2010 • Grid Rework • Improving Ease of Use • Organizational Status • View Ahead

17. Improving Ease of Use • Reduced open bug reports by 25% and support requests by 50%. • Robustness and standardization improvements, including: • Fixed memory leaks • Implemented standard treatment of real and integer data types, initialization of variables, and standard error handling framework-wide • Standard treatment of optional arguments in the C interface, contributed by Tim Campbell/NRL Stennis • ESMFI3 Wiki installed, outlined two levels of compliance • Interface compliance (for 2010) • Metadata compliance (beyond 2010) • New website

18. Metrics: Bugs

19. Metrics: Support Requests

20. Metrics: Feature Requests

21. New Project Website • New material: FAQ, metrics, values, impacts, regression test results, more • http://www.esmf.ucar.edu

22. ESMF_ArraySparseMatMul() Performance Grid is 720x360 to 1080x640. Courtesy Peggy Li/NASA JPL.

23. Outline • Achievement Highlights • Development Plan Through 2010 • Grid Rework • Improving Ease of Use • Organizational Status • View Ahead

24. ESMF Team (red is remote) Manager – Cecelia DeLuca, administration, QC Operations Manager – Sylvia Murphy, computing, space, website, tools Administrative Assistant (as available)– Jennifer Tobyne Integrator – Silverio Vasquez, test lead Project Scientist – David Neckels, unstructured grids and regrid Project Scientist – Don Stark, tutorials, use test cases, and test framework Developer – Bob Oehmke, structured grids Developer – Gerhard Theurich, arrays, communications, virtual machine, components, build Developer – Fei Liu, fields and bundles of fields Developer – Rosalinda de Fainchtein, C interfaces and standardization Developer (half-time) – Peggy Li, optimization Developer (half-time) – Gary Block, testing Developer, Julien Chastang – Component database (Curator project) Student Intern, Ryan O’Kuighttons – attributes and metadata To be hired: Project Scientist (since August, 2007) Developer (since September, 2007) Gaps: utilities (IO, time, LogErrr), standardization, documentation, and QC, attributes and metadata

25. ESMF Organizational Summary Working Project: Day-to-Day Operation • Core Team – team responsible for ESMF software implementation and user support, located at NCAR • Change Review Board (CRB) – multi-agency board that meets quarterly via telecon to set development priorities (DoD member: Alan Wallcraft) • Joint Specification Team (JST) – ESMF users who participate in weekly telecons that cover design reviews, code reviews, and other technical topics Executive Management • Executive Board – body that sets high-level direction and controls the structure of the ESMF organization (DoD member: Edward Franchi) • Advisory Board – offers general guidance to the Executive Board (DoD member: Steven Payne) • Interagency Working Group (IAWG) – group of program managers that provides programmatic direction and advocacy (DoD participant: Andrew Mark)

26. ESMF Governance ExecutiveManagement Executive Board Strategic Direction Organizational Changes Board Appointments annually Reporting Interagency Working Group Stakeholder Liaison Programmatic Assessment & Feedback Advisory Board External Projects Coordination General Guidance & Evaluation Reporting Working Project Change Review Board Development Priorities Release Review & Approval Joint Specification Team Requirements Definition Design and Code Reviews External Code Contributions quarterly Functionality Change Requests weekly Resource Constraints Implementation Schedule Collaborative Design Beta Testing Core Development Team Project Management Software Development Testing & Maintenance Distribution & User Support daily

27. Outline • Achievement Highlights • Development Plan Through 2010 • Grid Rework • Improving Ease of Use • Organizational Status • View Ahead

28. View AheadEnd-to-end workflows and automation Sample: • Browse model components and check compatibility • Download model components and input data • Configure and run model / assimilation • Publish output and archive data products • Analyze and visualize output • Verify against observational data • Annotate and archive results

29. View AheadEnd-to-end workflows and automation Component catalog and science portal • Working with Community Data Portal (CDP) and Earth System Grid (ESG) through the Curator project • Adapt CDP/ESG ontology for model assembly: components, models, configured models • Develop UI for CDP for component/model browse, query, download, upload • Embed standard metadata in ESMF data structures and enable in/out • Sanity check-level compatibility check for component sets • Future: automatic coupler and application generation Compliance verification • Metadata compliance definition and checks Looser coupling • Creating shared objects of components so they can be compiled separately • Automatically generated web service interfaces so ESMF components can be used in a range of SOAs

31. ESMF Organizational Goals • Prioritize development tasks in a manner acceptable to major stakeholders and the broader community, and define development schedules based on realistic assessments of resource constraints (CRB) • Deliver a product that meets the needs of critical applications, including adequate and correct functionality, satisfactory performance and memory use, ... (Core) • Support users via prompt responses to questions, training classes, minimal code changes for adoption, thorough documentation, ... (Core) • Encouragecommunity participation in design and implementation decisions frequently throughout the development cycle (JST) • Leverage contributions of software from the community when possible (JST) • Create frank and constructive mechanisms for feedback (Adv. Board) • Enable stakeholders to modify the organizational structure as required (Exec. Board) • Coordinate and communicate at many levels in order to create a knowledgeable and supportive network that includes developers, technical management, institutional management, and program management (IAWG and other bodies)