The Earth System Grid (ESG)

1. The Earth System Grid (ESG) Collaborations and Visibility DOE SciDAC ESG Project Review Argonne National Laboratory, Illinois May 8-9, 2003

2. Presentation Agenda • Why Collaborate? • Collaboration Framework and Partners • Summary of ESG Collaborations Earth System Grid

3. Part I Why Collaborate?

4. Why Collaborate? • Enterprise information sharing and management helps in productivity • Coordination of effort to avoid the duplication of work (i.e., to avoid re-inventing the wheel, save money, time and resources) • To bring together shared development of products useful to the entire community (e.g., Linux OS) • Reach common Goals that will appeal to a wider audience • It promotes visibility in the community and a wider acceptance of ESG components • Promotes consistency and standards throughout the community Earth System Grid

5. How can Collaborations Help? • Keeping team members up to date • Promoting access and retrieval of data suitable to user needs • Ensuring quality • Continuity • Minimizing biased code decisions and making code more efficient and effective • Meaning looking out to the community for better ways and avoid becoming insular Earth System Grid

6. Goals of ESG Collaborations • Leverage the work of others to meet ESG goals • Technology and operation experts and partner to develop and maintain community components • Develop usable and re-usable products • Usability means easy to learn, effective to use and provide an enjoyable experience for the user/developer • Merge ESG technology with the external community by providing a wide-range of Grid-enabled tools • Involve users in the design process • To make ESG more useful to climate researchers by interacting with potential ESG users Earth System Grid

7. Motivation and Strategies for ESG Collaboration • Collaborate across teams and organizational boundaries with customers/partners • Move outside the realm of ESG in decision making • Find the correct information quickly and easily • work on it collaboratively, and • publish it centrally online for use and re-use • Improve the inter-operability between the diverse national and inter-national groups and agencies • Ensure that everyone on the project has up-to-date, reliable information and understands exactly what that information means to ESG and the community Earth System Grid

8. ESG Outreach: Community Events and Interactions • ECMWF Computing Workshop • Earth System Modeling Framework Workshop • NOMADS Workshops • High Performance Network Planning Workshop • SciDAC Visualization Workshop • Workshop on Data Provenance and Derivation • Kickoff Meeting, NCSA Alliance PACI Data Quest Expedition • National Institute for Environmental eScience (NIEeS) Launch Workshop • British Atmospheric Data Center (BADC) • NIEeS Workshop on Metadata and Ontology • Aceess Grid meetings with the SciDAC Climate group and the U.K. BADC and CLRC teams • CODATA 2002 18th International Conference • DOE Townhall Meeting Earth System Grid

9. Part II Collaboration Framework and Partners

10. Collaboration Framework Analysis Tools Mgmt Services Data Portal Web Services Arch. and Technologies Storage and Networking • Disparate model • Coordinated development processes • Shared technologies ESG Climate Modeling Resource Sharing and Provisioning Globus Grid Technologies Distributed Data Mgmt Infrastructure Earth System Grid

11. Globus Toolkit Storage Resource Management (SRM) Middleware Project OPeNDAP (DODS): Distributed Oceanographic Data System (Unidata) Integrations of Globus GridFTP THREDDS: framework for publishing, cataloging, describing scientific datasets NcML - Co-development CDAT: Climate Data Analysis Tools (PCMDI) NCL: NCAR Command Language LAS: Live Access Server (NOAA Pacific Marine Environmental Laboratory) Works with CDAT, NCL We leverage our own community, and the world. Climate Community Grid Community Earth System Grid

12. Collaborations Partners • CCSM Data Management Group • Other SciDAC Projects: Climate, Security & Policy for Group Collaboration, Scientific Data Management ISIC, & High-performance DataGrid Toolkit • CEOS-grid • e-Science, UK • Council for the Central Laboratory of the Research Councils (CLRC) • Natural Environment Research Council (NERC) DataGrid • Earth Science Portal (ESP) group (multi-agency, inter-national) • Earth System Modeling Framework (ESMF) • NOAA Operational Model Archive and Distribution System (NOMADS) • Center for Ocean-Land-Atmosphere (COLA) Studies Earth System Grid

13. e-Science (CLRC and NDG): Collaboration Example • ESG team and the e-Science team met via the AG for a collaboration work strategy team meeting • Common Goals • Interoperability between systems • Proposted Tasks • Describe each group’s metadata • Overlaping areas • Fill in gaps in metadata • Merge CLRC, NDG, and ESG metadata where appropriate and introduce higher-level abstract concepts between elements and build an ontology for the Earth Sciences Earth System Grid

14. OPeNDAP: Leverage and Collaboration Example … Application Layer • OPeNDAP Striped Server = OPeNDAPg • OPeNDAPg Client Application Library • Climate applications (e.g., NCL, CDAT) builds with OPeNDAPg Client Application Library • Client Application Access GridFTP data via OPeNDAPg CDAT Client NCL Client Presentation Layer Data Local? remote Data Grid-enabled netCDF library OPeNDAP server On the Grid? OPeNDAP striped server module OPeNDAP Transport Layer ncgrid client striped GridFTP server Globus libraries ESG Component Contributions Earth System Grid

15. Collaboration Partnership Considerations • Ownership of software and data • Open Source • Security • Access to software and data • Authentication • Funding the work that must be done • Maintenance of software and data repositories • Liability • Software and data accuracy Earth System Grid

16. Lessons Learned • Embrace and leverage complementary activities • Keep the collaboration simple • Focus on core needs • Ensure a “win-win” collaborative effort • Facilitate constant feedback • Be open to giving all a chance • Don’t compete - facilitate • Meet the overall mission and purpose of ESG Earth System Grid

17. Part II Summary of ESG Collaborations

18. Summary of ESG Collaborations • National and International collaboration • Develop methods, standards, and software components integral to the success of ESG and usable to the larger Earth Science community • Diverse internal structure (Metadata Groups, Grid and Network Groups, Data Portal Groups, Modeling Groups, and Data Visualization Analysis Groups) • Provide unique contributions to the Earth Science communities Earth System Grid

19. Summary of ESG Collaborations • A Future challenge will be to maintain efficient and effective communication between all ESG collaborations. Earth System Grid