The European DataGrid Project

1. The European DataGrid Project Fabrizio Gagliardi EU DataGrid Project Leader CERN fabrizio.gagliardi@cern.ch

2. Supercomputer, PC-Cluster Data-storage, Sensors, Experiments Internet, networks The Grid metaphor Mobile Access G R I D M I D D L E W A R E Workstation Visualising Fabrizio Gagliardi EDG5 Piliscsaba

3. Technology Trends • Very fast networks can be built and are becoming much cheaper • Archival and disk storage becoming much cheaper • Sensors and instruments becoming ubiquitous and have much higher resolution => oceans of data • Computing power continues to increase • so many groups can analyze lots of data • Middleware, while still developing, is not vaporware • Everyone uses the web, so concept is not foreign • Some demonstration projects have shown the benefits (EU DataGrid test bed among others) Fabrizio Gagliardi EDG5 Piliscsaba

4. Grids Leverage Major Investments in Experimental Research Facilities • Research instruments, satellites, particle accelerators, etc., cost a great deal • Data from those devices can be accessed and analyzed by many more scientists from anywhere • not just the team that gathered the data or institutes from privileged countries • Easier international collaboration Fabrizio Gagliardi EDG5 Piliscsaba

5. The EU DataGrid Project • 21 Partners • 9.8 M Euros EU funding over 3 years • 90% for middleware and applications (HEP, EO and Biomedical) • Three year phased developments & demos (2001-2003) • Spin-off: • DataTAG (2002-2003) • CrossGrid (2002-2004) • GridStart (2002-2004) Fabrizio Gagliardi EDG5 Piliscsaba

6. EU DataGrid Project Objectives • Use Grid technology to develop a sustainable computing model for effective share of computing resources and data for large scientific communities • Specific project objectives: • Middleware for fabric & Grid management (mostly funded by the EU) • Large scale testbeds (mostly funded by the partners) • Production quality demonstrations (partially funded by the EU) Fabrizio Gagliardi EDG5 Piliscsaba

7. EU DataGrid Project Objectives • Collaborate with and complement other European and US projects • Test and demonstrator of EU RN/Geant • Contribute to Open Standards and international bodies: • Co-founder of Global GRID Forum and host of GGF1 and GGF3, co-founder of GRIDSTART • Dissemination • Industry and Research Forum for dissemination of project results • Education: Tutorials, CSC02, Road Show Fabrizio Gagliardi EDG5 Piliscsaba

8. Project Schedule • Project started on 1/1/2001 • TestBed 0 (early 2001) • International test bed 0 infrastructure deployed • TestBed 1.0 ( March 2002 ) • Project successfully reviewed by EU on March 1st 2002 • TestBed 1.2 ( now ) • First release of EU DataGrid software to defined users within the project: • HEP experiments, Earth Observation, Biomedical applications • TestBed 2 (end of 2002) • Builds on TestBed 1 to extend facilities of DataGrid • TestBed 3 (Fall 2003) • Project completion expected by end 2003 Fabrizio Gagliardi EDG5 Piliscsaba

9. DataGrid Main Partners • CERN – International (Switzerland/France) • CNRS - France • ESA/ESRIN – International (Italy) • INFN - Italy • NIKHEF – The Netherlands • PPARC - UK Fabrizio Gagliardi EDG5 Piliscsaba

10. Assistant Partners • Industrial Partners • Datamat (Italy) • IBM-UK (UK) • CS-SI (France) • Research and Academic Institutes • CESNET (Czech Republic) • Commissariat à l'énergie atomique (CEA) – France • Computer and Automation Research Institute,  Hungarian Academy of Sciences (MTA SZTAKI) • Consiglio Nazionale delle Ricerche (Italy) • Helsinki Institute of Physics – Finland • Institut de Fisica d'Altes Energies (IFAE) - Spain • Istituto Trentino di Cultura (IRST) – Italy • Konrad-Zuse-Zentrum für Informationstechnik Berlin - Germany • Royal Netherlands Meteorological Institute (KNMI) • Ruprecht-Karls-Universität Heidelberg - Germany • Stichting Academisch Rekencentrum Amsterdam (SARA) – Netherlands • Swedish Research Council - Sweden Fabrizio Gagliardi EDG5 Piliscsaba

11. EDG Highlights • The project has passed the first EU review (last March) • All 21 partners have contributed at least at contractual level • All EU deliverables (40, >2000 pages) submitted in time (available, code & documents via www.edg.org) • 60 EU funded FTEs and 106 un-funded FTEs • One of the largest Grid technology development and demonstrator • Real applications being deployed now Fabrizio Gagliardi EDG5 Piliscsaba

12. Particle Physics Applications CMS Storage – Raw recording rate 0.1 – 1 GByte/sec ATLAS Accumulating data at 8-10 PetaBytes/year 10 PetaBytes of disk LHCb Processing – 100,000 of today’s fastest PCs Fabrizio Gagliardi EDG5 Piliscsaba

13. Data mining on genomic databases (exponential growth) Indexing of medical databases (Tb/hospital/year) Collaborative framework for large scale experiments (e.g. epidemiological studies) Parallel processing for Databases analysis Complex 3D modelling Biomedical applications Fabrizio Gagliardi EDG5 Piliscsaba

14. Earth Observations • ESA missions: • about 100 Gbytes of data per day (ERS 1/2) • 500 Gbytes, for the next ENVISAT mission (launched March 1st) • EO requirements for the Grid: • enhance the ability to access high level products • allow reprocessing of large historical archives • improve Earth science complex applications (data fusion, data mining, modelling …) Fabrizio Gagliardi EDG5 Piliscsaba

15. Testbed Sites (>40) Dubna Moscow Lund Estec KNMI RAL Berlin IPSL Prague Paris Brno CERN Lyon Santander Milano Grenoble PD-LNL Torino Madrid Marseille BO-CNAF HEP sites Pisa Lisboa Barcelona ESRIN ESA sites Roma Valencia Catania Francois.Etienne@in2p3.fr - Antonia.Ghiselli@cnaf.infn.it DataGrid Testbed Fabrizio Gagliardi EDG5 Piliscsaba

16. Project Dissemination www.eu-datagrid.org CERN School of Computing 2002 • Programme includes • Grid Lectures by • Ian Foster • Carl Kesselman • Hands-on tutorial • DataGrid Vico Equense, Italy, 15-28 September 2002 The 2002 CERN School of Computing is organised by CERN, with the Institute of Composite and Biomedical Materials, National Research Council, Naples, Italy Fabrizio Gagliardi EDG5 Piliscsaba

17. Future Plans • Concentrate on production quality and real applications • Complete the programme of work till end of 2003 • Educate new users and disseminate results • Make plans to conserve momentum and assets in the EU FP6 Fabrizio Gagliardi EDG5 Piliscsaba

18. A proposal to the EU • EDG has already demonstrated the viability of this technology • EU had a fast start in supporting 18 Grid projects in the last two years! • RN Geant offers an excellent basis for a large European Grid infrastructure • The EU FP6 programme should encourage and support the deployment and production quality operation of a large international Grid infrastructure open to research and industry in Europe Fabrizio Gagliardi EDG5 Piliscsaba

19. The EGEE EoI • EGEE: Integrated Infrastructure Initiative (III) to support ERA (European Research Area) 300 M € planned (100 M € requested) • Vision: to create and deploy Grid technologies to enable the widespread uptake of e-Science applications throughout the European Research Area • Four key objectives: • integrating Grid technological developments from across Europe; • establishing a Europe-wide Grid infrastructure for science and industry with a focus on heterogeneity and interoperability; • enabling the creation of e-Science applications from across the scientific and industrial spectrum; • ensuring the timely delivery of the project’s programme of work, guided by the needs of academic and industrial partners. Fabrizio Gagliardi EDG5 Piliscsaba

20. The EGEE EoI continued • Approach: coordinated submission with RN Geant follow-up (200 M € requested) • Focus on the Grid production infrastructure to support vertical and horizontal applications (vertical: Bio, EO, PP, Engineering, Commerce? Industry? horizontal: CS and added value services) • Special emphasis of the EU programme on SMEs • Distributed managerial structure by coordinating national and regional Grid initiatives Fabrizio Gagliardi EDG5 Piliscsaba

21. Final word… • EU DataGrid is a good first step together with the other related Grid initiatives • Need not to lose momentum but aggregate interest and resources in science, industry and funding agencies for proposals in the EU FP6 • Significant contribution from the Hungarian partners so far • Important step this week here to review the progress and make plans for the future Fabrizio Gagliardi EDG5 Piliscsaba

22. Many thanks for the kind hospitality and excellent support! Fabrizio Gagliardi EDG5 Piliscsaba