1 / 10

Data management in SCD Steven Worley

Explore the SC Data Management and Research Archives, home to over 211K primary data files and 530 datasets in various sizes. Access data, metadata, and project descriptions through the web-based Information Server.

amycsmith
Download Presentation

Data management in SCD Steven Worley

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data management in SCDSteven Worley General Categories • The Mass Storage System • NCAR user file services (home directories) • Computer attached storage networks • SCD Research Data Archives • 20 Terabytes

  2. Characteristics • Development began in middle 1960’s, (35 years) • Managed by seven people • 211K primary data files • 530 datasets – all sizes

  3. Search and Discover Data • How?  Web based Information Server • Features • 5K+ html pages (metadata) • All datasets are described • Location of all data files in MSS • Access options are identified • Higher level information • Catalogs • Project specific descriptions Always current dataset descriptions

  4. Dataset Page • Title and Brief description • Systematic Navigation • Metadata highlights • Period of Record • Usage • Variables • Related Sites (NOAA) • Contact Person • Related Datasets

  5. Archive Content • Weather Center Operational Analyses • NCEP beginning in 1976 • Many products – global, regional, different resolutions, full atmosphere stack, all surface grids • ECMWF beginning in 1980 • Ditto • Atmospheric Reanalyses • NCEP/NCAR 1948 – 2001 • NCEP Version II, 1979 – 2000 • ECMWF ERA15, 1979 – 1993 • ECMWF ERA40, 1958 – 2002 • NCEP N.America Reanalysis

  6. Outstanding Features • Three different coordinate surfaces • Very long analysis, 2+ Terabytes size • Unrestricted distribution • CD-ROMS are very popular

  7. Archive Content • Observational dataset • Comprehensive Ocean-Atmosphere Data Set (COADS), beginning in 1794 • Global land surface observations, beginning in 1930’s, some earlier • Global upper air measurements, beginning in 1946 • Many more supporting datasets and analyses

  8. Real time access to data • Community Data Portal • Multi-tiered systems • Gui, middleware, file service (local or remote) • Access + subsets + analysis • Much more from Don • Some SCD Research Datasets will be served through CDP

More Related