High Data Volume Transfer Issues at NOAA

1. High Data Volume Transfer Issues at NOAA Christopher D. Elvidge Earth Observation Group National Oceanic and Atmospheric Administration National Geophysical Data Center Boulder, Colorado August 26, 2005 chris.elvidge@noaa.gov

2. NOAA’s Vision and Mission • VISION – An informed society that uses a comprehensive understanding of the role of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions. • MISSION – To understand and predict changes in the Earth’s environment and conserve and manage coastal and marine resources to meet our Nation’s economic, social, and environmental needs.

3. Terrestrial, Coastal & Marine Ecosystems Water Resources Natural & Human Induced Disasters Energy Resources Sustainable Agriculture & Desertification Human Health & Well-Being Biodiversity Weather Information, Forecasting & Warning Climate Variability & Change Benefits of Satellite Earth Observations

4. NOAA’s Current Satellite Observing Systems • Polar-orbiting Operational Environmental Satellites (POES) ~10 GB per day. • Geostationary Operational Environmental Satellites (GOES) ~ 10 GB per day. • Defense Meteorological Satellite Program (DMSP) ~ 8 GB per day.

5. NOAA’s Future Satellite Observing Systems • NPOESS Preparatory Project (NPP) ~4 TB per day 2008- 2012. • National Polar-orbiting Operational Environmental Satellite System (NPOESS) C1 ~4 TB per day 2010-2015. • National Polar-orbiting Operational Environmental Satellite System (NPOESS) C2 ~4 TB per day 2012-2017. • National Polar-orbiting Operational Environmental Satellite System (NPOESS) C3 ~4 TB per day 2014-2019. • Geostationary Operational Environmental Satellites-R (GOES-R) ~ 2 TB per day 2012-2017.

6. Projected NOAA Data Holdings By Year

7. The NOAA Comprehensive Large Array-data Stewardship System(CLASS) will be the long term archive for the data from these systems, providing data access for the scientific community.

8. Hardware and Network Design • CLASS will have three operational sites (NGDC, NCDC, and NASA IV&V) • CLASS will have a centralized development environment and a centralized integration and test environment at NSOF • CLASS will use a Multi-protocol Label Switching (MPLS)-based peerless IP network used for site to site communication • New hardware and network architecture • Implemented by January 2006 • Modular, scalable, and redundant • Addresses security requirements

9. CLASS Multi-Site Configuration

10. CLASS Distribution Process

11. NGDC will have direct access to the NLR via the Front Range GigaPop for use in delivering data from CLASS

12. The Data Hog? • The Visible-Infrared Imaging Radiometer Suite (VIIRS) will fly on NPP and each of the NPOESS satellites. • Raw data (RDRs) from a VIIRS will run 116 GB per day. • Radiance calibrated geolocated data (SDRs) will tilt to 815 GB per day. • Environmental data records (EDRs) will bulge towards 2 TB per day.

13. The Challenges • Moving the data within NOAA - from the Suitland to the CLASS nodes. • Providing simple, useable tools to establish and manage large volume data subscriptions. • Transferring large volumes of data to the users, including those in the APAN region. • MAFFIN, Chiba U., U. Tokyo, Tokoku U. in Japan have expressed interest in receiving global VIIRS data!

14. Thanks for ListeningQUESTIONS?