150 likes | 301 Views
EOSDIS Core System Architecture. 10/21/2009 ESDSWG Dan Marinelli. EOSDIS Context. EOSDIS Context. EOSDIS Core System Architecture. ECS is responsible for the ingest, archive and distribution of NASA earth science data
E N D
EOSDIS Core System Architecture 10/21/2009 ESDSWG Dan Marinelli
EOSDIS Core System Architecture • ECS is responsible for the ingest, archive and distribution of NASA earth science data • Deployed to three sites, Land Processes DAAC, National Snow and Ice Data Center, and Atmospheres Science Data Center • Supports data management for 9 instruments on 7 spacecraft • Primary stakeholders are: DAACs, SIPS, ECHO Providers (non-ECS), Instrument Teams, Science Users, End Users, ESDIS Project
Current Capacities, etc. • Online Archive volume (current/Jan 2010) • LP ~1PB/1.13PB; NSIDC 98TB/108TB; ASDC 632TB/739TB • Ingest Rates (daily requirement - granules/TB) • LP 111k/1.68; NSIDC 48k/.2; ASDC 36k/2 • Distribution Rates (daily requirement - granules/TB) • LP 428k/5.2; NSIDC 178k/1; ASDC 73k/5.4 • Development facility has 634 TB of test data contained in ~120M granules • Currently, ECHO has over 15000 users searching/ordering data
Other Notes • The Science Data Processing Toolkit (SDPTK) is installed at 15 IT facilities in the US, Japan, and Europe • Metadata Time and Date tools separated out for teams that just need to write metdata • Algorithm Theoretical Basis for the toolkit algorithms was published in 1995!
Communications (CSS) aInternetworking (ISS) ECHO BMGT (Export Metadata & Browse) PULL PUSH ECHO Orders Data Management DMS Data Pool Ingest (DPL INGST) CLS (Check Order Status) Order Manager (OMS) External Data Sources and Providers SSS Online Archive/ Data Pool (DPL) DSS Insert and Retrieve Science Software and Data Tape Archive (LTO) EMOS ECS Subsystem Architecture Diagram
ECS Subsystem Summary • BMGT - exports inventory status information • CLS - user interface to check order status. • DMS - providing support for data retrieval across all ECS sites. • DPL supporting the search, order, and distribution of selected granules with associated metadata and browse granules (if available).
ECS Subsystem Summary (cont.) • DPL INGEST service will handle the SIPS ingest interface, S4P, cross-DAAC ingest, EDOS ingest, ASTER Ingest and Polling without Delivery Record specifically for EMOS. • DSS with the functions needed to manage the inventory of archived data.
ECS Subsystem Summary (cont.) • OMS managing all orders received from the DMS EWOC (i.e., from WIST and ECHO), the Machine-to-Machine Gateway, and the Spatial Subscription Server. • SSS supporting the creation, viewing and updating of subscriptions and the creation, viewing and deletion of bundling orders (specification of distribution packages and criteria for package completion).
ECS/ECHO References • http://edhs1.gsfc.nasa.gov/ • Design docs, operations procedures, SDPTK user guides, HDF-EOS reference and user guides • For ICDs and other ESDIS-controlled documentation contact me at daniel.j.marinelli@nasa.gov • For ECHO documentation contact andrew.e.mitchell@nasa.gov, or if you’re brave go to our EED RFP website: http://esdis.eosdis.nasa.gov/eed/index.html and find what we’ve posted there (for a limited time!)