1 / 14

IT discussion

Explore challenges and solutions in enhancing data flow efficiency for Earth science research collaboration at the InSAR Working Group Meeting. Discussion on software, hardware, standards, security, and user needs.

rwiggin
Download Presentation

IT discussion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IT discussion Day 1 afternoon breakout session http://solidearth.jpl.nasa.gov/insar InSAR Working Group Meeting, Oxnard, Oct 2004

  2. Archive Manager Catalog DLR-GSOC Ground Station Weilheim (WHM) DLR-GSOC Ground Station Weilheim (WHM) University of Miami Alaska SAR Facility (ASF) Alaska SAR Facility (ASF) MIT NSIDC NSIDC Stanford University NSIDC Caltech EDC Permanent Archive SDSC Long Term Archive 5 years near-line User GDS Data Flow User Acq. Request 300 Mb/sec downlink Status to User Mission Control Center (MCC) Mission Control Center (MCC) Mission Control Center (MCC) Internet - 2 Transfer to NTS • Network Transfer Subsystem (NTS) • Stanford University & NSIDC • Archive Manager • Network Transfer Subsystem (NTS) • Stanford University & NSIDC • Archive Manager JPL Mission Ops JPL Mission Ops JPL Mission Ops Populate ST Archive Short Term Archive 1 year on-line Internet - 2 Update Catalog Stanford University Caltech MIT NSIDC Howard University SDSC Long Term Archive 5 years near-line EDC Permanent Archive User Data. Request Populate Permanent Archive in Background User Redirect L1 Data Internet and Internet-2 Archive Manager Archive Manager Archive Manager Catalog Catalog Catalog Catalog Catalog Catalog redirects user to appropriate FTP address for L0 data retrieval Data FTP Collection requests, acquisition requests, data requests Populate LT Archive In Background User User User User Science Team Acquisition plan, status, holdings, software, ancillary data SIO Science Planning Bd. SIO Science Planning Bd. InSAR Working Group Meeting, Oxnard, Oct 2004

  3. SERVO … Goddard Ames JPL Tier2 Center Tier2 Center Tier2 Center Tier2 Center Tier2 Center HPSS HPSS HPSS HPSS … Observations Solid Earth Research Virtual Observatory (SERVO) Archive • 1 PB per year data rate in 2010 • Distributed Heterogeneous Real-Time Datasets ~TBytes/day … Archive … … Downlink … Downlink Tier 0 +1 HPSS Archive … … Downlink 100 TeraFLOPs sustained Tier 1 Tier 2 Fully functional problem solving environment Tier 3 • Program-to-program communication in milliseconds • Approximately 100 model codes Institute Institute Institute Institute 100 - 1000 Mbits/sec Data cache Tier 4 Workstations, other portals • Plug and play composing of parallel programs from algorithmic modules • On-demand downloads of 100 GB in 5 minutes • 106 volume elements rendering in real-time InSAR Working Group Meeting, Oxnard, Oct 2004

  4. National Lambda Rail (NLR) SEA POR SAC BOS NYC CHI OGD DEN SVL CLE WDC PIT FRE KAN RAL NAS STR LAX PHO WAL ATL SDG OLG DAL JAC 15808 Terminal, Regen or OADM site Fiber route Transition beginning now to optical, multi-wavelength Community owned or leased fiber networks for R&E NLR • Coming Up Now • Initially 4 10G Wavelengths • Full Footprint Ops by 3Q or 4Q04 • Internet2 HOPI Initiative (w/HEP) • To 40 10G Waves in Future • Regional Dark Fiber Initiatives in 18 U.S. States InSAR Working Group Meeting, Oxnard, Oct 2004

  5. Solution: Move to Internet Protocol Over Dedicated Optical Lightpaths Coupling NASA Centers to NSF OptIPuter NASA Goddard IT Pathfinder Working Group Earth and Climate Scientists- Creating a Virtual Collaboratory Between Goddard and SIO (UCSD) InSAR Working Group Meeting, Oxnard, Oct 2004

  6. Field Trip Data Database ? GISGrid Discovery Services RepositoriesFederated Databases Streaming Data Sensors Database Sensor Grid Database Grid Research Education SERVOGrid Compute Grid Customization Services From Researchto Education Data FilterServices ResearchSimulations Analysis and VisualizationPortal EducationGrid Computer Farm InSAR Working Group Meeting, Oxnard, Oct 2004 Geoscience Research and Education Grids

  7. Basic System Architecture InSAR Working Group Meeting, Oxnard, Oct 2004

  8. Schematic of a Typical Ontology Manager InSAR Working Group Meeting, Oxnard, Oct 2004

  9. What will be needed as technology for: • Access to data through data bases and the web • Software for data mining and data exploration • Software for simulating, interpreting, visualizing data InSAR Working Group Meeting, Oxnard, Oct 2004

  10. Who are the End Users outside the InSAR Research Community? • Engineering • Education • Government/Policy decision makers • Operational/Hazards • Commercial InSAR Working Group Meeting, Oxnard, Oct 2004

  11. Issues Associated with Access to data through databases • Standards for domestic and international missions • Security • Quality and Validation • Bandwidth • Interoperability – XML & GML • Content addressable storage • Near real time or real time • Rapid response tasking of satellite InSAR Working Group Meeting, Oxnard, Oct 2004

  12. S/w and H/w • Data exploration and analysis • Higher level functionality is desirable • Portal architecture • Peer to peer • Collaborative • Error quantification • Graduated authentication • S/w development • Curation • Reliability and robustness • Fusion InSAR Working Group Meeting, Oxnard, Oct 2004

  13. Simulating, interpreting, and visualizing data • Large scale simulation & modeling capability is needed & should be part of mission plan • Centralized and Grid computing resources • Code sharing (open source) • Code validation and curation • Reliability and robustness InSAR Working Group Meeting, Oxnard, Oct 2004

  14. Remaining questions • Commercial component? • Operational versus Science mission? InSAR Working Group Meeting, Oxnard, Oct 2004

More Related