1 / 16

Earth Science GRID at ESA GRID on-Demand, e-collaboration…

Earth Science GRID at ESA GRID on-Demand, e-collaboration…. Luigi Fusco, Pedro Pereira Gonçalves. Summary. > 1 Terabyte data processed – 1 Km res. Earth Science and GRID Present infrastructure at ESRIN Example of applications Medium term vision and plans. GRID for Earth Science means.

nova
Download Presentation

Earth Science GRID at ESA GRID on-Demand, e-collaboration…

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Earth Science GRID at ESA GRID on-Demand, e-collaboration… Luigi Fusco, Pedro Pereira Gonçalves

  2. Summary > 1 Terabyte data processed – 1 Km res. • Earth Science and GRID • Present infrastructure at ESRIN • Example of applications • Medium term vision and plans

  3. GRID for Earth Science means Cluster Operating System Intelligent Interface System Users Supercomputing Networking Mass Storage Middleware • Enhance the ability to create high level products • Single stop shop for data access and processing • Support PIs in new science developments • Allow processing of large historical archives • e-collaboration, • e.g. sharing of data sources, tools, means, models, algorithms … • improve Earth science complex applications (data fusion, data mining, modeling …)

  4. EO GRID Infrastructure Status • Infrastructure • Computing and Storage Elements: • One GRID LAN for ops and dev environment • 2 clusters (64, 16),~20 PCs, 20+10+5 TB • Still using GLOBUS 2.4 and LCG (EGEE software) • Link to external CE and SE • Data access: • ENVISAT Rolling Archive (very stable) and ESRIN AMS (not fully stressed) • Access to NASA and other external data providers • Network: • GIGAbit DATAGRID LAN • 40 GIGA bit link in progress ( > May 2006) • Security: • ESACERT agreement • Centralized user certification from ESA

  5. Transfer to Operation

  6. Grid on Demand Portal Services • Temporal/spatial selection of data from ESA catalogue • Job selection, chaining, launching & live status • Visualization in Web Map • Access to L3 products • Access to reference doc

  7. Online Storage • MERIS • MERIS Level 2: 2003- to date~6Tb • MERIS FR (samples) ~100Gb • MERIS Level 3 • AATSR • AATSR Level 1: 2003-2005 ~9Tb • AATSR Level 2: last 6 months ~300Gb • Atmospheric • GOME Level 1 + Ozone profiles: 1996-2003 ~1Tb • GOMOS: June 2003 – April 2004 ~300Gb • Radar • ASAR GM: Feb 2004 – to date ~300Gb • ASAR High and Medium Resolution (samples) ~700Gb • MSG • SEVIRI: Just started (last 6 months) ~3TB (planned) • AVHRR, SeaWiFS • Under discussion @ CNR ~ 2TB (planned)

  8. Applications for routine operations… • Web Mapping MERIS L3 Layers for Promotion • Paper under discussion with H.Laur • NRT MERIS L3 Ocean/Atmosphere • Under final validation (end Oct) • Complement L3 sets of ACRI • NRT MERIS L3 Vegetation (MGVI) • Joint collaboration with JRC (N Gabron) • Ready for operation (end Sept) • Processed historical archive of Europe for validation (article in MERIS/AATSR WS)

  9. SAR Applications… • NRT Antarctica ASAR GM mapping • Daily Generation of 400-m Mosaic • Automatic publish in ESAWeb Map (C.Caspar) • Retrieval of NASA ice concentration contour (end Sept) • Ready for operation • Large scale ASAR/SAR mapping • Reuse of BEST functions • Dedicated RF (R Cossu) • To be developed in coming months • RESPOND interest for Systematic Flood Monitoring • Compatible with MERIS mapping on GRID

  10. Other internal Collaborations • MERIS L3 Quality checks • Flags and aggregation methods • Effect of Solar Angle and Glint for Algal-1 • Operated autonomously by P.Goryl • VOMIR-Volcano monitoring by InfraRed • Integrated and operated autonomously by O.Colin • Enables fast validation procedures (1 volcano-> 3 years of data in dispersed Storages -> 1 hour processing in 20 CPU) • Collaboration w/ Kings College (M.Wooster)

  11. Volcano Monitoring integration in Google

  12. More Internal Collaborations • Atmospheric chemistry (K Zehner, R Kopman) • Stratospheric aerosol density processing (BIRA-IASB, Belgium) • 3D data assimilation system • Water Vapor and Temperature (DMI, DK) • ASAR Flood Monitoring (P. Bally) • Radar Altimeter Individual Echoes and S Band Study (J.Benveniste) • Study of GODIS Integration with EOLI (O Barrois) • GlobCarbon GRID study by VITO (O Arino) • MSG products (S Pinnock / B Greco) • Earthcare, SMOS simulations … (ESTEC)

  13. Additional External Collaborations Mainly based on CAT-1 project approval • MERIS applications: case 2 waters,chlorophyll, vegetation (SISCAL, …) • Objective analysis SST and chlorophyll in the Med (CNR, ENEA) • Integration of AATSR/MERIS/MSG for fire detection (CNR…) • Reprocessing of ERS-2 SAR for oil spill statistics (UniTV) • Direct access to MODIS AQUA&TERRA from NASA archives (CEOS) • more…

  14. Summary of Grid ES applications • Some 40+ applications supported by GRID at ESRIN • Some ready for operations • Integration of ESA developed toolboxes (BEAM, BEST, BEAT…) • Ongoing internal and external collaborations  some to be considered for transfer to operations later • Activities based on GRID team resources • Reference report under preparation • CAT-1 approach to allow data access is working properly • More Earth Science applications in collaboration with EGEE infrastructure (EC) • More if additional support resources are made available

  15. Companion GRID projects • GRID – EC funded: • Participation to EGEE (follow-on DataGrid) • DILIGENT– Digital Libraries in GRID environ’t • started in Sept 04 • ESA in charge of Environmental User scenario • e-collaboration – EC funded: • BRAINBRIDGE - started in Mar 05 • define road map for e-collaboration initiative in FP7 • Infrastructure – Regional funding: • MEGALAB • GRID infrastructure across Frascati-Tor Vergata-Rome area (to be started soon) • ESA projects: • PECS-GRID: SAR application- started in Sept 05 • The VOICE: e-collaboration cal/val • New proposals for EC funding under preparation

  16. Conclusion • Operational Opportunities • Routine generation of “standard” high level products • On demand GRID products from User Services • Extend Grid infrastructure to facilities (e.g. Kiruna) • Increase exploitation of available data/resources • More explicit support to PIs • Extend JRC collaboration experience • Prepare a dedicated GRID AO – support 10 key users/y • Move more tools (BEAT, BEST…) in the grid environment and consider next EO missions • Enable new classes of large-scale, high-power applications • Prepare the future • Participation of Earth Science to the EC e-infrastructure initiative • Contribution to GEOSS architecture

More Related