Summary of the EOSDIS Reference Architecture

1. Summary of the EOSDIS Reference Architecture ESIP Federation Meeting ESDSWG SPG July 22, 2010

2. Outline • Background • EOSDIS Architecture Framework • Science Data System Functions • Cross-cutting Functions • Scenarios • Satellite Data • Field Campaign • Airborne • Issues

3. Flight Operations, Data Capture, Initial Processing, Backup Archive Data Transport to Data Centers/ SIPSs EOS Spacecraft EOSDIS Context Science Data Processing, Data Mgmt, Interoperable Data Archive, & Distribution Data Acquisition Distribution and Data Access Tracking & Data Relay Satellite (TDRS) Research Education EOS Data and Operations System (EDOS) Data Processing White SandsComplex(WSC) EOSDIS Data Centers Value-Added Providers Internet (Search, order, distribution) Interagency Data Centers EOS Operations Center (EOC) Mission Control DirectBroadcast(DB) Earth System Models International Partners NASA Integrated Services Network (NISN) Mission Services EOS PolarGround Stations Instrument Teamsand Science Investigator-ledProcessing System (SIPS) Decision Support Systems Direct Broadcast/ Direct Readout Stations ESDIS ESMO

4. Data Center SIPSs EOSDIS Facilities Data centers, collocated with centers of science discipline expertise, archive and distribute standard data products produced by Science Investigator-led Processing Systems (SIPSs) ASF DAAC SAR Products Sea Ice, Polar Processes SEDAC Human Interactions in Global Change CDDIS Crustal Dynamics Solid Earth NCAR, U of Col. HIRDLS, MOPITT, SORCE LP DAAC Land Processes & Features GSFC GLAS, MODIS, OMI, OBPG GES DISC Atmos Composition & Dynamics, Global Modeling, Hydrology, Radiance Key NSIDC DAAC Cryosphere, Polar Processes OBPG Ocean Biology & Biogeochemistry JPL MLS, TES ORNL DAAC Biogeochemical Dynamics, EOS Land Validation LAADS/ MODAPS Atmosphere LaRC CERES, SAGE III San Diego ACRIM GHRC Hydrological Cycle & Severe Weather PO.DAAC Ocean Circulation Air-Sea Interactions ASDC Radiation Budget, Clouds, Aerosols, Tropo Chemistry GHRC AMSR-E, LIS

5. Purpose • Purpose of the EOSDIS Reference Architecture • Documents the way EOSDIS does business, in all of its variations • Enables discussion with other communities and agencies on earth science data system development • It provides a framework for the insertion of community developed systems into the core system • Informs new missions of the current processes • Serves as a starting point for a future-looking Earth Science Data System Architecture

6. Discipline and mission specific data systems Only community-specific standards Only directory functions to point and link users to various data systems Multi-site access and data inter-use cumbersome Improved access to data with 90s technology Cross-system search and order access via interoperability data model Site-specific interfaces at DAACs/SIPs Common distribution format (HDF); other formats also supported Evolution of EOSDIS Data System Features Technology, Funding, Lessons Technology, Funding, Lessons <1990 mid-1990s

7. Support for high data volumes, ambitious performance requirements, high availability/reliability Integrated core infrastructure plus loosely coupled elements from community Common data model Automated metadata creation and ingest No need for cross-site metadata translation FGDC standards compliance Expanded set of software tools and services Flexible options for supporting or interoperating with external data sources Coexistence of heterogeneous, distributed data providers/ information partners Optimal set of core standards Support for community-specific standards Reusable software components Exploitation of Service Oriented Architecture On-line archives and cross-system service invocation Ease of innovation and technology infusion More services to meet community demands Evolution of EOSDIS Data System Features Technology, Funding, Lessons Technology, Funding, Lessons Technology, Funding, Lessons Late 90s to present Near-Future

8. Core and Community Capabilities - Characteristics

9. EOSDIS Architecture Framework

10. Satellite Data Scenario #1

11. NASAEDOS JAXA-EOC L1A Product Generation Ground Stations NSIDC DAAC Data Archive and Distribution AMSR-E Data Flow RBD & PDS Level-0 NASAFDS GBAD Level-1 Level-1 PO.DAAC Level-1 SIPS-RSS TLSCF Level-2 and Level-3 L2A algorithm L2A Product Generation Science Software I&T L2 and L3 DAPs Product Delivery Record Server (PDRS) SIPS-GHCC L2B and L3 Algorithms Algorithm Teams L2B and L3 Product Generation (science, metadata, qa, ph, subsets, browse, DAPs) Rain Snow Data Ocean Sea Ice Land

12. Descriptions of EOSDIS data holdings can be found at the archiving data center. See AMSR-E at the National Snow and Ice Data Center (NSIDC). Select the Order Data option. http://nsidc.org/data/amsre/

13. Users have a variety of ways to access EOSDIS data. For example select the WIST option. http://nsidc.org/data/amsre/order_data.html

14. Warehouse Inventory Search Tool (WIST) • The EOS ClearingHouse (ECHO) is a metadata catalog of NASA’s EOS data and a registry for related data services (e.g. reformatting, pattern recognition). ECHO’s catalog contains more than 2800 data sets held at 12 EOSDIS data centers. • WIST, the Warehouse Inventory Search Tool, is the primary web-based client for discovering and ordering cross-discipline data from all of ECHO’s metadata holdings. WIST allows users, including those without specific knowledge of the data, to search science data holdings, retrieve high-level descriptions of data sets and detailed descriptions of the data inventory, view browse images, and submit orders via ECHO to the appropriate data providers. • The WIST search-and-order tool has the following features:  • Search capability—Allows a user to explore the list of data sets or granules returned by a search by viewing their temporal coverage, spatial coverage, attributes (metadata), related documents (guide search), and browse images. • Order function—Allows the user to select data for ordering, choose packaging information, enter ordering information (such as shipping address), place an order, and view order status. Reference: http://wist.echo.nasa.gov/~wist/api/imswelcome/

15. Other Scenarios • Field Campaign (In situ) scenario • Ground and Ocean-based scenario • Airborne scenarios • Ancillary scenarios • Satellite scenarios • US • International

16. Examples of EOSDIS Data Center Implementations

17. EOSDIS Reference Architecture • Plan both a document & website • EOSDIS Reference Architecture Draft • Under discussion and drafting components • DAACs • Committee of NASA data system planners • Scenarios Developed • Satellite Data • Field Campaign • Airborne • Basic Website in place • However, the reference architecture only documents EOSDIS functions in a very broad approach • Does not take the place of official NPR documentation • Each new Mission requires direct communication and distinct documentation

18. Example Science Interfaces & Documentation ESDIS Project IPA Flight Project ICD W/A Science Team ESDIS Project DMR ADURD ATBD ICD ICD IRD ICD OA EDOS Data Centers SIPSs ICD ICD Users ICD ICD ICD ICD ECHO A subset of interfaces are shown for mission operations Networks ESDIS Metrics System ESMO Project MRT Mission Readiness Test Plan http://esdis.eosdis.nasa.gov/dsdocs

19. Interoperability on Many Levels • Process Methodology • Configuration control • Business processes • Policy processes • Software Reuse – S4PA, subsetting • Standards • Metadata model • Data formats • Interface controls • EOSDIS ClearingHouse • EOSDIS Metrics components

20. EOSDIS Business Model • Budget preparation and tracking of actual expenses is performed across the Project. • Components of EOSDIS submit yearly budgets and work plans • Budgets and actuals are tracked monthly against phasing plans and schedules • Metrics are collected for both technical and fiscal management • Coordination across the architecture allows for cost savings

21. Specific Areas of cost tracking

22. Conclusion • The EOSDIS Reference Architecture is still under draft by many staff members • Thanks to all of you!!! • Any questions? Comments? • Feel free to contact me: • Jeanne Behnke • jeanne.behnke@nasa.gov

23. AMSR-E – Advanced Microwave Scanning Radiometer - Earth Observing System JAXA – National Space Development Agency (Japan) TLSCF – Team Leader Science Computing Facility (AMSR-E) RSS – Remote Sensing Systems (company) GHCC – Global Hydrology and Climate Center (MSFC, Huntsville, Ala) DAAC – Distributed Active Archive Center PO.DAAC – Physical Oceanography DAAC NSIDC – National Snow and Ice Data Center (NSIDC) DAP – Delivery Algorithm Package GBAD – Ground based attitude determination FDS – Flight Dynamics System RBD – Rate buffered data PDS – Production data set