1 / 25

Data Discovery and Access to The International Surface Pressure Databank (ISPD)

Data Discovery and Access to The International Surface Pressure Databank (ISPD). Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley National Center for Atmospheric Research, Boulder, CO *NOAA/CIRES, Boulder, CO. AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2.

jacob-gross
Download Presentation

Data Discovery and Access to The International Surface Pressure Databank (ISPD)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Discovery and Access to The International Surface Pressure Databank (ISPD) Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley National Center for Atmospheric Research, Boulder, CO *NOAA/CIRES, Boulder, CO AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  2. Research Data Archive (RDA) at NCAR rda.ucar.edu 600+ distinct datasets for climate and weather research Collections: ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs Free and open access http://rda.ucar.edu AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  3. ISPD Overview • World’s largest collection of surface & sea level pressure observations • Land station • Marine observations (ICOADS) • Tropical cyclone best track (IBTrACS) • Period (version 2): 1768 – 2010 • Volume: 465 Gbyte • Available since Aug 2010 AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  4. ISPD Overview • 60+ Contributors • Atmospheric Circulation Reconstructions over the Earth (ACRE) • Australian Bureau of Meteorology • British Antarctic Survey • Cook Islands Meteorological Service • Danish Meteorological Institute • DeutscherWetterdienst (DWD; German Weather Service) • European and North Atlantic Daily to Multidecadal Climate Variability (EMULATE) • ETH Zurich, Switzerland • GCOS/WCRP Working Group on Observational Data Sets for Reanalysis • MANY MANY MORE…. • Assembled by NOAA/ESRL, CIRES (Univ. of Colorado), & NOAA/NCDC AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  5. ISPD Sample Annual Station Distribution 1850 * Land stations only * No marine stations AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  6. ISPD Sample Annual Station Distribution 1850 * Land stations only * No marine stations 1900 AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  7. ISPD Sample Annual Station Distribution 1850 * Land stations only * No marine stations 1900 1950 AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  8. ISPD Sample Annual Station Distribution 1850 * Land stations only * No marine stations 1900 1950 2000

  9. ISPD Observations/Year 2010: 53 Million ~ 1.5 Billion total observations Figure courtesy Chesley McColl, NOAA/ESRL

  10. Data Access: Problem Background Scientific data centers have these resources Individual researchers generally don’t • Large computational/storage resources needed • Store data • Extract desired data from large grids/files • Convert data to desirable format(s) AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  11. Data Access: Problem Background • Researchers stay focused on research • Goals • Make data more accessible and easier to use for individual researchers • Reasonable access volumes • Desired data formats • User defined parameters/grids AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  12. ISPD Data Access Services • Powerful computing resources @ NCAR • Large disk storage (~ 0.5 PB) • Rich and detailed metadata • Direct file download via web • Customized data sub-setting • HDF-5 to ASCII software tools (NetCDF forthcoming)

  13. ISPD Metadata Features • Both group- and file-level metadata • Drive interfaces for file grouping and sub-setting tools • Support efficient back-end processing • Improve scalability • Provide “quick look” at data samples AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  14. ISPD Metadata Interface Example AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  15. AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  16. AGU 2012 Fall Meeting: IN44A-05

  17. AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  18. Data Access: ISPD Subset Interface AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  19. ISPD Data Access Services Temporal range sub-setting (daily) • Spatial sub-setting • Lat/Lon region • Individual station ID

  20. ISPD Data Access Services • Data sub-setting options (cont.) • Observation type • Land station • Marine obs • Radiosonde • Dropsonde • TC best track

  21. ISPD Data Access Services • Subsetting processed in delayed mode • E-mail notification • Download via server-provided scripts (wget)

  22. ISPD 2012 Subset metrics Data accessed: ~ 6.5 TB Data served: ~ 46 GB AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  23. Summary & Future Directions • RDA – Supply “User Friendly” Data • Parameter & spatial sub-setting • Metadata discovery • Format conversion • Improved and additional services • NWSC-Cheyenne opening – more computing power AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  24. ISPD Forthcoming • DOI assignment • Geoscience Data Journal article • ISPD v3(1755-2010) • Spring 2013 AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

  25. rda.ucar.edu/ds132.0 tcram@ucar.edu AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2

More Related