1 / 44

Zaihua Ji, NCAR Steven Worley, NCAR Scott Woodruff, NOAA/ESRL

Data to Support Ocean-Atmosphere Research NCAR Research Data Archive (RDA), http://www.dss.ucar.edu. Zaihua Ji, NCAR Steven Worley, NCAR Scott Woodruff, NOAA/ESRL Fourth International Ocean-Atmosphere Conference July 4 - 6, 2007, Qingdao, China. Presentation Outline. Introduction Overview

odessa-valenzuela
Download Presentation

Zaihua Ji, NCAR Steven Worley, NCAR Scott Woodruff, NOAA/ESRL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data to Support Ocean-Atmosphere ResearchNCAR Research Data Archive (RDA),http://www.dss.ucar.edu Zaihua Ji, NCARSteven Worley, NCARScott Woodruff, NOAA/ESRL Fourth International Ocean-Atmosphere Conference July 4 - 6, 2007, Qingdao, China

  2. Presentation Outline • Introduction • Overview • Focus, Data Relevant to Ocean-atmosphere Research • RDA Data Archive and Management • RDA Data Usage Statistics • Explore Data Archived in RDA • Future Developments • Summary

  3. Introduction Definition, Research Data Archive (RDA) • Dataset collection to support atmospheric and related geosciences research • Hosted by the Computational and Information Systems Laboratory (CISL) at National Center For Atmospheric Research (NCAR) • Built with data curation and stewardship over more than 40 years • Presently maintained by eight people, educated in meteorology and oceanography • Basic Scale metrics • + 500 datasets • + 800K files • + 120 TB

  4. Introduction Why RDA Access is Unrestricted • Our user community is best served if no restrictions exists • Research needs a very large array of data types • Dataset development requires experts - no one can do it all • Many studies require global data - everyone can make a contribution and everyone should benefit. • Data Management alone has significant associated costs (people, storage, servers, etc) • There are near zero funds to purchase data • If service systems are well designed many people can be economically served • Foremost • We know data sharing works well and we want to be a “best practices” example in the international research community • We have had successful data collaborations and exchanges with China • Focused on building and improving ICOADS

  5. Overview - Data Relevant to Ocean-Atmosphere Research Marine observations • International Comprehensive Ocean-Atmosphere Data Set (ICOADS) • Marine surface observations from ships, moored and drifting buoys, and fixed platforms • Very long time series, 1794-2005, over 200 million obs. • Merging and QA/QC of many different datasets • 20 + year collaborative project between NOAA and NCAR, now receiving more international collaboration • More assistance is welcomed • RDA provides two product lines • Observations • Monthly summary statistics in 1x1 and 2x2 lat. lon. boxes • ICOADS is a source for higher level analyzed products, e.g. • SST analyses • Global atmospheric reanalyses

  6. Overview - Data Relevant to Ocean-Atmosphere Research ICOADS • Example of ICOADS and productsSST: Dec 1942ICOADS and two analyses HadISST from UKMO ERSST from NOAA HadISST ERSST

  7. Overview - Data Relevant to Ocean-Atmosphere Research • ICOADS monthly summary statistics • 10 statistics, e.g. median, mean, nobs… • 23 observed and derived parameters • no bias corrections • All source observations are available in a similar way - derive your own statistics

  8. Overview - Data Relevant to Ocean-Atmosphere Research • World Ocean Database (WOD) and World Ocean Atlas (WOA) • Prepared by Ocean Climate Laboratory NODC, Sydney Levitus and collaborators • First edition 1982, now at 2005 • Worlds largest collection WOD • Approximately, 1900 and forward • Ocean profile observations from many international sources • Many types (XBT, CTD, ARGO, bottle stations, etc) • Quality checked and on observed and standard depth levels Profiling float data added WOD01  WOD05

  9. Overview - Data Relevant to Ocean-Atmosphere Research WOA • analyzed data on standard levels, 1 degree resolution • climatology, seasonal, monthly products (T, S, O2, nutrients, etc..) • Some special products • annual heat content • mix-layer depth • high resolution (.25 degree) T and S climatology

  10. Overview - Data Relevant to Ocean-Atmosphere Research Marine Analyses - SST NOAA OptimumInterpolation SST Analysis • 1981- ongoing , weekly, monthly, 1°x 1°, updated monthly • ICOADS, real-time obs., AVHRR sat. blend • (May ‘07 in background) NOAA Extended Reconstructed SST Analysis • 1854 - ongoing, monthly, 2°x 2°, updated monthly • ICOADS and real-time obs. Hadley Centre HadISST • 1850 - ongoing , monthly, 1°x 1°, updated monthly • ICOADS, real-time obs., AVHRR sat. blend for modern period NOAA Optimum Interpolation Daily SST Analysis • 1/4 °x1/4 °, 1985 - ongoing, two products, updated monthly • ICOADS, real-time obs, AVHRR satellite • ICOADS, real-time obs., AVHRR and AMSR satellite

  11. Overview - Data Relevant to Ocean-Atmosphere Research Marine Analyses - Wind • Global QSCAT/NCEP Blended Analysis • Created by Colorado Research Associates • 0.5° x 0.5 °, 4x daily, 1999 - ongoing • Vector wind components, and curl • New versions are prepared when QSCAT algorithms are improved. Currently, a new version is under development. • Many other good sources from Atmospheric Reanalysis datasets • To be presented later

  12. Overview - Data Relevant to Ocean-Atmosphere Research Marine Analyses - Waves • Global and North Atlantic Atlas of Ocean Waves • Created by Sergey Gulev • 2° x 2 ° and 5° x 5° , monthly, 1958 - 1997 • Significant wave height • Derived from ICOADS visual wave observations • ERA-40 • 1.5° x 1.5 °, 4x daily, 1957 - 2002 • 2-D wave spectra are available for 12 directions and 25 frequencies

  13. Overview - Data Relevant to Ocean-Atmosphere Research • Global Objectively Analyzed Air-Sea Fluxes (OAFlux) • Created by Yu and Weller • 1° x 1°, daily and monthly • Latent and sensible flux, 1981-2002, net short and long wave radiation 1983-2002 • Derived from in situ observations, satellite data, and NWP and reanalysis output • Update is planned for 2003 and onward • Ocean-atmosphere heat, momentum and freshwater flux • Created by Josey et al. • 1° x 1°, climatology • Derived from in situ observations and adjusted to match ocean heat transport estimates Marine Analyses - Fluxes

  14. Overview - Data Relevant to Ocean-Atmosphere Research • Sources • Many useful surface products at different resolutions. Too many to cover here • Main dataset title and temporal coverage • NCEP/NCAR Reanalysis, 1948 - ongoing • NCEP/DOE Reanalysis, 1979 - ongoing • ERA-40, 1957-2002 • JRA-25, forthcoming from JMA, 1979 - ongoing • MERRA, forthcoming from NASA, 1979 - ongoing • Important note about ERA-40 • By agreement RDA can distribute only in U.S., and to a few universities in Canada. • Copy outside this domain is prohibited Marine Analyses - Atmospheric Reanalyses

  15. Overview - Data Relevant to Ocean-Atmosphere Research • NCEP Global Ocean Data Assimilation System (GODAS) • 1° x 1°, pentad and monthly, 1980 - ongoing • 1/3 ° resolution ± 10° of equator • 40 vertical levels, parameters (T, S, ocean current, etc) Marine Analyses - Ocean Reanalyses GODAS Annual mix layer depth

  16. Overview - Data Relevant to Ocean-Atmosphere Research • NCEP Global Final Analysis (FNL) • 1° x 1°, 4x daily, 1999 - ongoing • Updated monthly • ECMWF Global Advanced Operational Surface Analysis • 1.125 ° x 1.125°, 4x daily, 1985 - ongoing • Updated bi-annually • TIGGE (THORPEX Interactive Global Grand Ensemble) • Analyses and forecasts from international NWP centers • Now CMA, ECMWF, NCEP, UKMO and JMA; will add 5 more centers • native model resolutions (approx. 1°), 1x to 4x daily, all ensemble members • Parameters; 10m winds, temperatures, surface radiation and fluxes, etc, • Update, near real time, 2-weeks data online NWP Operational Output

  17. RDA Data Archive and Management • RDA Data Archive Components • Creating Dataset Metadata • Populating Datasets with Data • Updating Datasets • Technology of data archiving • Accumulation of data archived in RDA

  18. RDA Data Archive Components

  19. Creating Dataset Metadata

  20. Populating Dataset with Data

  21. Updating Datasets

  22. Archiving Data with State of Art Technology • Hybrid archiving system with Database Management System (DBMS), Database for short, and file management system • Comprehensive archiving utility: gathering data, archiving to MSS and/or online service, recording file metadata, and publishing data filelists • Automated and complete for periodic data updates: downloading, processing/validating, and archiving data • All archiving, web and DBMS utilities are based upon open source software • Both onsite and offsite backup of whole RDA

  23. Data Archived In RDA(MSS: 121.5 TB, Online: 14.9 TB; By 2007-05)

  24. RDA Data Usage Statistics • How data are accessed • What data are accessed • Who uses the RDA data

  25. How Data Are Accessed(Downloaded between 2006-06-01 & 2007-05-31)

  26. How Data Are Accessed(Between 2006-06-01 & 2007-05-31)

  27. Top 10 Datasets Accessed by Most Users(between 2006-06-01 & 2007-05-31)

  28. Top 10 Datasets by Download Volume(between 2006-06-01 & 2007-05-31)

  29. Who uses RDA Data(Accessed Between 2006-06-01 & 2007-05-31)

  30. Top 20 Countries/Areas by Most Users(Online data Between 2006-06-01 & 2007-05-31)

  31. Top 10 Datasets by Most Users FromP.R.China, Taiwan & Hong Kong(Online Data Between 2006-06-01 & 2007-05-31)

  32. Monthly Online Users FromP.R.China, Taiwan & Hong Kong(Between 2005-01 & 2007-05)

  33. Monthly Online Data by Users From P.R.China, Taiwan & Hong Kong(Downloaded Between 2005-01 & 2007-05)

  34. Explore Data Archived in RDA • Registration • Identify dataset • Get the data • Online data • MSS Filelist • Subsetting • Questions and special requests

  35. Register Yourself on RDA Server

  36. Identify Datasets You Want

  37. How to Access Data For A Specified Dataset

  38. How to Access Data For A Specified Dataset

  39. Access Online Data … …

  40. MSS File List … …

  41. Data Subsetting … …

  42. Future Developments • New data search tool based on standardized metadata, GCMD • Browse tool to identify the files needed, based on DB and file metadata queries • Expanded RDA content, e.g. JRA-25, more ocean reanalysis • User feedback and information sharing at the dataset level using web forum technology • More user subsetting and format conversion • More open archiving and access service for NSF projects • Need specific inclusion and management policies • Modernization of web page look and function

  43. Summary • NCAR’s RDA is an open international data resource • We encourage and facilitate data exchanges • We continually build and improved access to the RDA • Modern DB systems are used for management and public data service • Modern data management methods (DB’s and automation) allow: • Systematic growth • Systematic and secure archiving • Accurate usage assessment metrics • We are always open to suggestions for new data resources and service improvements from the users

  44. Thank You Questions? Contact: • http://dss.ucar.edu/index.html • Zaihua Ji, zji@ucar.edu

More Related