APDRC...Quo Facis?

1. APDRC...Quo Facis? Data serving and archiving at the Asia-Pacific Data-Research Center IPRC

2. The APDRC was established within the IPRC to be a climate data and web-based product serving facility. Our mission is to increase understanding of climate variability in the Asia-Pacific region by: • developing the computational, data-management, and networking infrastructure necessary to make data resources readily accessible and usable by researchers and other users; • undertaking data-intensive research activities that will both advance knowledge and lead to improvements in data collection and preparation.

3. Challenges for Distribution of Climate Data • disparate data types • station data, gridded data, ocean data, atmospheric data, levels, time-series, etc. • disparate data formats • netcdf, grib, flat binary, etc. • datasets can be large • coupled model out, long integrations, high resolution (time/space), etc. • users needs are widely varying

4. Data Managementand Data Serving • metadata (so-called “readme” files) • data discovery --> search for data • data mining • data transport --> getting data to users • OPeNDAP, ftp, http, etc. • on-line browse --> view data • LAS • data archive

7. Data Transport • ftp (limited) • direct binary access NFS (internal) • OPeNDAP

8. DODS/OPeNDAP • DODS: Distributed Oceanographic Data System, a.k.a OpenDAP: Open Source Project for a Network Data Access Protocol, is a software framework used for data networking. • DODS provides tools for making local data accessible to remote locations regardless of local storage format

9. On-line Browse • LAS (gridded data) • EPIC (station data)

10. Live Access Server (LAS) • LAS is a Web server designed to provide flexible access to geo-referenced scientific data. LAS enables the Web user to visualize data with on-the-fly graphics • request custom subsets of variables in a choice of file formats • access background reference material about the data (metadata) • compare (difference) variables from distributed locations

11. Serving gridded products

14. Compare distributed GODAE model outputs Define region Select time

15. Serving non-gridded products

16. Data Archive • parallel systems (Linux and Solaris based) • similar raid systems • remote aggregation • data spread across ocean, atmosphere, air-sea flux • model reanalysis, forecast, etc.

17. Old paradigm User access to data either by copy or ftp, then use local software Data reformat (netcdf, ieee, grib, etc.)? Client s/w limitations? Disk storage/transfer limitations?

18. Non-gridded (station) data Gridded data

19. “sister server”

20. GrADS Matlab ferret

21. APDRC Project Support • Global Ocean Data Assimilation Experiment (GODAE) • argo • Pacific Regional Integrated Data Enterprise (PRIDE) • Regional modeling • Ministry of Education, Culture, Sports, Science and Technology (MEXT) • JAMSTEC Earth Simulator Group (ESG) • Quality control of historical profiles of temperature and salinity for the global oceans (HydroBase2, WHOI) and the Indian Ocean (CSIRO), GTSPP

22. GODAE As the international community participates in the demonstration phase of GODAE, (2003-2005) there is an overarching need for rapid delivery of data products from satellites and models to the broad user community including regional operational entities.

23. GODAE • Operational Systems • Naval Oceanographic Office (NAVOCEAN; NRL, Bay St. Louis, MS) • Fleet Numerical Meteorology and Oceanography Center (FNMOC; Monterey, CA) • National Centers for Environmental Prediction (NCEP) • Research and Development Systems • Argo • NASA Seasonal to Interannual Prediction Project (NSIPP) • Estimating the Circulation and Climate of the Ocean (ECCO; Scripps, MIT and JPL) • Hybrid Co-ordinate Ocean Model (HYCOM; Miami, NRL) • Naval Research Laboratory (NRL) • Data • National Virtual Ocean Data System (NVODS)

24. EPIC access to argo data Select subregion Sort function

25. Further subset

26. Plot specific station Plot/save along section

27. PRIDE Conceptual Framework forPRIDE US/NOAA Context Bi-Laterals with Australia Japan New Zealand Global/Regional Observations GCOS/GOOS/IOOS IPRC APDRC Pacific Services Center New NOAA Facility in Hawaii Pacific Climate Information System WMO/NOAA RCCs RISA NOAA Data Centers

30. Future thoughts;what does this mean for CLIVAR? • Users • Requirements? • Data discovery, data transport? • Regional analysis of global models • Collaborators • production of data products • communication of results to wider audience • include model output as data products? • metadata

31. Issues, cont’d • Who does this? • Key is to determine users and uses • Model interpretation; data providers=data users? • Development of data products • Leave details to someone else?

32. Non-scientific users: Plot generation More co-ordinated approach Web interface >> data format/server tech APDRC Customers Scientific users: • direct access to data • typically large (long timeseries and/or large spatial extents Server technology >> interface/presentaion

33. TBA • Development/identification of data products • Use of numerical models • Future observational programs • Model metrics • Use of observations

34. The World Climate Research Program (WCRP) was formed in 1980 by the International Council for Science (ICSU) and the World Meteorological Office (WMO) to provide international framework for scientific cooperation in the study of global change. The Global Climate Observing System (GCOS) was established in 1992 to ensure that the observations and information needed to address climate-related issues are obtained and made available to all potential users. GCOS is sponsored by the WMO, the ICSU, and the Intergovernmental Oceanographic Commission (IOC). The Global Ocean Observing System (GOOS) is a sustained, co-ordinated international system for gathering data about the oceans and seas of the Earth, a system for processing such data, with other relevant data from other domains, to enable the generation of beneficial analytical and prognostic environmental information services, and the research and development on which such services depend for their improvement. In 1978 (?), WCRP, IOC and the Scientific Committee on Oceanic Research (SCOR) formed the Climate Change Committee for the Ocean (CCCO). This committee then established the Ocean Observation Panel for Climate (OOPC). The OOPC oversees the implementation of an ocean observation system for GCOS (or, equivalently, the climate component for GOOS) and, with GCOS the Atmospheric Observation Panel for Climate (AOPC). OOPC has several projects, including CLIVAR, GODAE and ARGO. GODAE therefore is a project of OOPC and is sponsored by GCOS, GOOS and WCRP.