1 / 20

AMRC Meteorological Data Distribution System Part I: Data Sources and Archive

AMRC Meteorological Data Distribution System Part I: Data Sources and Archive. K. Elena Willmot, Matthew A. Lazzara, Linda M. Keller, Nicole M. Schroeder Antarctic Meteorological Research Center Space Science and Engineering Center University of Wisconsin-Madison. Introduction.

rodney
Download Presentation

AMRC Meteorological Data Distribution System Part I: Data Sources and Archive

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AMRC Meteorological Data Distribution System Part I:Data Sources and Archive K. Elena Willmot, Matthew A. Lazzara, Linda M. Keller, Nicole M. Schroeder Antarctic Meteorological Research Center Space Science and Engineering Center University of Wisconsin-Madison

  2. Introduction • AMRC since 1992... • Part I Outline: • Data Sources Internal & External • Data Formats • Data Archive Historical & Present • Distribution • Challenges No more FTP? Access and quality of data Lack of acknowledgements • Statistics (Photo by Michael Forster Rothbart, UW-Madison University Communications)

  3. Data Sources: Internal • AWS (Automatic Weather Station) data • Significant portion of internal data comes from UW’s AWS network • Real-time and archive quality data sets • Ten minute intervals • Hourly intervals • Three-hourly intervals • 60+ weather stations across the continent

  4. Data Sources: Internal • Antarctic Composites • Conceived by UW’s Professor Charles R. Stearns in October 1992. • Cover South Pole region to about 40° South. • Satellites used: GOES, GMS, MTSAT, METEOSAT, FY-2, KALPANA, NOAA, DMSP, Aqua & Terra, and MetOp • Created every 3 hours/1 hour every day • Currently, 4 sizes: • 512 x 512 • 683 x 683 • 1024 x 1024 (standard) • 2048 x 2048 • 6 types: • Infrared • Water Vapor • Experimental Visible • Long-wave (Coming soon!) • Short-wave (Coming soon!) • Pseudo Color 

  5. Making Antarctic Composite Satellite Imagery:

  6. Data Sources: Internal • University of Wisconsin’s Space Science and Engineering Center (SSEC) Data Center • Surface synoptic observations • METARs (Meteorological Aviation Reports) • RAOB (Radiosonde Observations) • Ship/buoy observations  • Numerical Models from GFS (Global Forecast System) and ECMWF (European Centre for Medium-Range Weather Forecasts). • Polar-orbiting LAC (Local Area Coverage) observations from NOAA satellites

  7. Data Sources: External • Participation from various researchers, USAP (United States Antarctic Program) participants, and international groups within the Antarctic meteorological community • International AWS (real-time only) • USAP research vessel observations • Field experiment radiosonde observations • Surface observations at: • Palmer Station • McMurdo Station • South Pole Station • Various other field camps Lawrence M. Gould. Photo by: Al Hickey http://photolibrary.usap.gov/

  8. Data Flow NOAA GAC/LAC SSEC Data Center MTSAT Meteosat (2) Products: GOES (2) FY-2C Aqua/Terra NASA/NOAA Kalpana-1 AMRC Office Wisconsin AWS International AWS McMurdo, Palmer & South Pole Stations USAP Research Vessels DMSP RTD Aqua/Terra DB (McMurdo only) NOAA HRPT

  9. Data Formats Archive: Distribution: McIDAS AREA MD GRID ASCII Text netCDF GIF, JPEG • McIDAS • AREA • MD • GRID • ASCII Text • GIF, JPEG

  10. Data Archive: Historical • Since 1992... • Media used to archive data: • Magneto-optic disks • Compact disks (CD) • Digital video disks (DVD) • 8 millimeter Exabyte tapes • DDS3 tapes (4 mm tapes) • 3590 IBM tapes • LTO tapes 3 1 2 6 4 5 7

  11. Data Archive: Present • RAID/5 disk arrays as well as DVDs are most prominent today • FTP site most widely used • Updated monthly with • AWS – SPAWAR obs • USAP research vessel observations • McMurdo Station obs • Palmer Station obs • South Pole Station obs • Summary / notification email sent out

  12. Distribution • Data Services to be discussed in Part II: • FTP (File Transmission Protocol) • Rsync Service • Antarctic-IDD (Internet Data Distribution) LDM (Local Data Manager) • ADDE (Abstract Data Distribution Environment) • E-mail • Website • Future: • RAMADDA (Repository for Archiving, Managing and Accessing Diverse Data) • GIS (Geographic Information Systems) Software

  13. Challenges:1 • USAP moving away from FTP site – advancing security rules • Don’t fret! FTP will still be available here at the AMRC: • Other options: Link with external system • Limited resources to maintain links • Manually providing data is expensive and not desired • We aim to provide data in a more semi-automated, self-service fashion.

  14. Challenges:2 • Accessing data externally • Poor quality • Corrupt files • Missing • Gaps in the archive • Which data can be used / distributed and which cannot? • Third party redistribution limitations • Evolving toward nearly fully automated data flow system (from source to archive) • Maintenance will be a constant issue – especially at the source • Documentation

  15. Challenges:3 - Metadata • Ever increasing need for Metadata • Significant effort to research/document and provide • Incomplete historical records, leave some metadata information missing… • E.g. South Pole...

  16. ANNOUNCEMENT! • We have found an issue with the South Pole data: • From 1996 and earlier, the South Pole LCDs (Local Climate Data) used data that switched between using local time and GMT time when recording observation times. • South Pole is working on re-running past LCDs and we do not know how long it will take / when they will be completed.

  17. Acknowledgements • The authors appreciate the support of the Automatic Weather Station Program and/or Antarctic Meteorological Research Center for the data set, data display, information, etc. ([AMRC/AWS staff member, as appropriate], NSF grants number ANT-0636873, and/or ANT-0838834) Sculptor Gabriel Warren used a transparency of the B-15A Iceberg in one of his pieces called “Interrogation: Dialogue” in his “Rhegmalogia” collection. http://www.art-farm.net/homeGabe.htm

  18. AMRC/AWS Statistics • Show Gigabytes transferred / month on Website. • Average of 70-80 Gb year-round. • Gigabytes transferred / month on FTP site. • Generally very variable – maybe due to field season.

  19. Please join our mailing lists!: • ANTIDD– Antarctic-IDD mailing list • ANTDISCUSSION – Antarctic meteorology discussion group • ANTAWS - Information and updates on AWS data • ANTOBS - Information and updates on all other observational datasets • ANTOPS - Notices on AMRC’s real-time data availability, distribution and status • ANTICEBERG - Notices on latest Iceberg monitoring images posted to the web site • ANTCOMMUNITY- Infrequent notices to the Antarctic community on events and significant developments How to subscribe: Visit our new website and click on the “Subscribe” button next to the mailing list you wish to join:

  20. Questions? Acknowledgements: National Science Foundation Office of Polar Programs Grant #0838834 South Pole Meteorology Office/Raytheon Polar Services SPAWAR Office of Polar Programs McMurdo Station Research Associate Palmer Station Research Associate USAP Research Vessels – LM Gould & NB Palmer McMurdo Ground Station International partners & More THANK YOU!

More Related