1 / 7

The eXtreme Data Cloud (XDC) project

The eXtreme Data Cloud (XDC) project is a 2-year, EU-funded initiative focusing on scalable technologies for federating storage resources and data management in distributed computing. It enhances existing Data Management services by harmonizing them in European e-Infrastructures. With a use-case approach, XDC integrates services like dCache, Dynafed, EOS, and FTS to improve Quality of Service and implement unified cross-federation data management. The project aims to support WLCG technologies, promote cost efficiency, and deploy a "Data Lake" style of deployment. Visit http://www.extreme-datacloud.eu/ for more info.

grantm
Download Presentation

The eXtreme Data Cloud (XDC) project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The eXtreme Data Cloud (XDC) project Oliver Keeble CERN Thanks to Daniele Cesinifor much of the material

  2. eXtreme Data Cloud • The eXtremeDataCloud is a 2 year, 3M€, EU-funded softwaredevelopment and integration project • Started work 1st Feb 2018 • Develops scalable technologies for federating storage resources and managing data in highly distributed computing environments • Improve already existing, production quality Data Management services • By adding missing functionalities requested by research communities • Must be coherently harmonised in the European e-Infrastructures INDIGO PaaS Orchestrator INDIGO CDMI Server XDC @ Napoli

  3. Represented research communities XDC @ Napoli

  4. A usecase driven project • New functionalities added on top of existing production quality services • Integration of different services to address the requirements XDC @ Napoli

  5. XDC topics • Large scale data management • dCache, Dynafed, EOS, FTS, Indigo Orchestrator • Quality of Service • Consistent implementation and management • Data Lifecycle management • Data Lakes • Multi-site storage and smart caching • Data Management Orchestration • + integration with compute • Unified cross-federation data management • Based on OneData XDC @ Napoli

  6. XDC “Data Lake” concepts cache.desy.de lake.infn.it dCache DataCentre 4 lake.cern.ch Federated access Cached access Custodial cloud.tsys.de EOS DataCentre 2 EOS Data centre 1 dCache DataCentre 3 Standalone xrootd/pure-HTTP/WebDAV cache Cloud Centre 5 Cached access Managed Buffer s3.amazon.com

  7. Summary • XDC is supporting development of a number of technologies used in WLCG • The objective is to enable the “Data Lake” style of deployment of interest to WLCG and to promote cost efficiency through QoS and smart data management. • Project currently ramping up, deploying test infrastructure, defining architecture and identifying first development goals. • http://www.extreme-datacloud.eu/ XDC @ Napoli

More Related