1 / 10

Work Package II - DIOS

Prototype a reliable and scalable federated data infrastructure to store, organize, and provision scientific data, enabling long-term preservation and integrating with commercial involvement. Partnerships with EU projects and ongoing initiatives. Work package tasks include infrastructure and federation services, orchestration, integration with compute services, networking, and authentication.

deborahm
Download Presentation

Work Package II - DIOS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Work Package II - DIOS Simone Campana CERN, Geneva, Switzerland Annecy, 7 February 2019

  2. Data Infrastructure for Open Science (DIOS) Goal: design, implement and operate a cloud of data services for open access and open science at the Exabyte scale The backbone of the Data Lake are well experienced large national data centers supporting the ESFRIs in ESCAPE The data lake will serve as underlying data infrastructure to manage and serve data to the user communities This solution will be proposed as key component of the future EOSC framework, supporting FAIR principles Simone.Campana@cern.ch

  3. Simone.Campana@cern.ch

  4. WP2- specific objectives Prototype a reliable and scalable federated data infrastructure • Stores and organizes scientific data (Findable) and enables the provisioning of data processing (Accessible) • Enables sciences to build open data repositories (Interoperable) • In general, supports the world-leading data challenges of the Research Infrastructures in ESCAPE Simone.Campana@cern.ch

  5. WP2- specific objectives Ensure long term data preservation (Reproducible) at the infrastructure level Archiving of data in certified repositories and capabilities to retrieve the data in the long term Complementing other work packages dealing with software, environment and provenance Simone.Campana@cern.ch

  6. WP2- specific objectives The Data Lake development leverages collaboration with and integrates the work from previous and ongoing frameworks • EU projects such as EOSC-hub and XDC • Initiatives at the Research Infrastructures • Ongoing work from e.g. GEANT, PRACE Simone.Campana@cern.ch

  7. WP2- specific objectives Computing Interface and Scalability • Computing capacity available inside and outside the data lake (Grid, Cloud, HPC, volunteer computing) • Need to integrate compute (and data) resources which are not part of the lake • Compute-data locality not guaranteed. Need to offer a reliable content-delivery service Simone.Campana@cern.ch

  8. WP2- specific objectives Industrial and Commercial involvement Commercial storage can be added to the data lake as cache or for resiliency Commercial computing can be integrated with the data lake as extra processing capacity In the data lake model, for both compute and storage there is no lock-in to the vendor WP2 will validate the use of commercial cloud to store and process scientific data Simone.Campana@cern.ch

  9. WP2 tasks Task 2.1 Data Lake Infrastructure and Federation Services. CERN (Xavier Espinal) Task 2.2 Data Lake orchestration service. DESY (Patrick Fuhrmann) Task 2.3 Integration with Compute Services. NOW-I-ASTRON Task 2.4 Networking. SKAO (Rosie Bolton) Task 2.5 Authentication and Authorization. INFN (Andrea Ceccanti) Simone Campana (CERN) as WP leader, Rosie Bolton (SKAO) as deputy Simone.Campana@cern.ch

  10. Final remarks about WP2 All WP tasks have well defined connections with other ESCAPE work packages: • WP1 for the link with the EOSC and to connect with the ESFRIs management boards • WP3 for the integration of WP2 services with the ESCAPE-EOSC service catalog • WP4 for the interface between the data lake and the astronomical science products data archives • WP5 to integrate the access to the data infrastructure from the analysis platforms Details about WP2 tasks, milestones, deliverables will be presented tomorrow To join the WP2 mailing list: https://listserv.in2p3.fr/cgi-bin/wa?SUBED1=e-dios To send an email to the mailing list: e-dios@ESCAPE2020.EU Simone.Campana@cern.ch

More Related