Fabrizio Gagliardi EU DataGrid Project Leader EGEE Project Coordinator f.gagliardi@cern.ch

1. Fabrizio GagliardiEU DataGrid Project LeaderEGEE Project Coordinatorf.gagliardi@cern.ch NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003 From Grid Prototypes to Grid Infrastructures EGEE is proposed as a project funded by the European Union under contract IST-2003-508833

2. Introduction • International computing networks are major enablers of new computing models • GRID computing is one of them, probably the most promising • Effective implementation of a truly distributed computing model across different non uniform computing administrative domains • Many prototype projects based on RN Geant and similar international initiatives NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 2

3. GriPhyN PPDG iVDGL Some DataGrid Projects Through collaborative Grid projects, there is the potential for a truely global scientific applications grid NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 3

4. EDG & GEANT • Connections of the different nodes of the EDG testbed are made possible by the EU-funded GEANT project • connecting more than 30 countries across Europe • speeds of up to 10 Gbit/s • high data throughput • Quality of Service • EDG and GEANT: the first major production quality tests of the network • speed • reliability • monitoring capabilities NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 4

5. Applications • Genomic Exploration • Earth Observation • High Energy Physics more and more scientists begin to use the Grid computing model and existing Grid testbeds,relying on Grid technology to solve huge data challenges NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 5

6. CMS ATLAS LHCb LHC – Resources Requirements Storage – Raw recording rate 0.1 – 1 GByte/sec Accumulating data at 5-8 PetaBytes/year 10 PetaBytes of disk Processing – 100,000 of today’s fastest PCs NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 6

7. DataGrid prototypes: DataGrid (I) • 9.8 M Euros EU funding over 3 years (twice as much from partners) • 90% for middleware and applications (High Energy Physics, Earth Observation, Genomic Exploration) • Total of 21 partners, over 150 programmers from research and academic institutes as well as industrial companies • Three year phased developments & demos (2001-2003) • Several improved versions of middleware software (final release end 2003) • Several components of software integrated in the large Particle Physics Production LHC Computing Project (LCG) • Software used by partner projects: DataTAG, CROSSGRID, GRACE NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 7

8. DataGrid prototypes : DataGrid (II) • DataGrid testbed: more than 1000 CPUs at more than 15 sites (up to 40) • Connections made possible by the EU-funded GEANT project • connecting more than 30 countries across Europe • speeds of up to 10 Gbit/s • high data throughput • quality of Service NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 8

9. The next step: go production • Similarly to research networks few years ago, after many prototype projects we need to go production • Major issues: Security, (im)maturity of M/Ware toolkits, difficult user interface for average use, cost and complexity of operations, etc… NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 9

10. EGEE vision:Enabling Grids for E-science in Europe • Goal • Create a wide European Grid production quality infrastructure on top of present and future EU RN infrastructure • Build on • EU and EU member states major investments in Grid Technology • International connections (US and AP) • Several pioneering prototype results • Larg Grid development team (>60 people in EDG) • Requires major EU funding effort • Approach • Leverage current and planned national and regional Grid programmes • Work closely with relevant industrial Grid developers, NRENs and US-AP projects Applications EGEE Geant network NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 10

11. Why EGEE? Impact on Society • Access to a production quality grid will change the way science and business is done in Europe An international network of scientist will be able to model a new flood of the Danube in real time, using meteorological and geological data from several centers across Europe A team of engineering students will be able to run the latest 3D rendering programs from their laptops using the Grid A geneticist at a conference, inspired by a talk she hears, will be able to launch a complex biomolecular simulation from her mobile phone NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 11

12. Why EGEE? Political context • Current Grid R&D projects run out within few months • The EGEE partners have already made major progress in aligning national and regional Grid R&D efforts, in preparation for EGEE • EGEE will preserve the current strong momentum of the European Grid community, and the enthusiasm of the hundreds of young European researchers already involved in EU Grid projects (>150 in EDG only) NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 12

13. Why EGEE? Historical analogy • Prior to the EU Geant programme, there was in Europe a multitude of exploratory projects in networking technology. Geant was truly production oriented, and brought European telecom operators actively into the picture • In a similar way, EGEE can ensure preservation of current investments in European Grid R&D, extending the present infrastructure and focussing all activities towards establishing a production quality Grid NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 13

14. EGEE Partner Federations • Integrate regional Grid efforts • Represent leading grid activities in Europe NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 14

15. EGEE Activity Areas • Services • Deliver “production level” grid services (manageable, robust, resilient to failure) • Ensure security and scalability • Middleware • Professional Grid middleware re-engineering activity in support of the production services • Networking • Proactively market Grid services to new research communities in academia and industry • Provide necessary education NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 15

16. Regional Support Regional Support Regional Support Resource Center Resource Center Resource Center EGEE Operations Structure Operations Center Infrastructure Regional Support Center (Support for Applications Local Resources) Resource Center (Processors, disks) Grid server Nodes NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 16

17. EGEE Service Activity • Create, operate, support and manage a production quality infrastructure • Structure: • EGEE Operations Management at CERN • EGEE Core Infrastructure Centres in the UK, France, Italy and CERN (leveraging HEP LCG at the start), responsible for managing the overall Grid infrastructure • Regional Operations Centres, responsible for coordinating regional resources, regional deployment and support of services in all other countries • Offered services: • Middleware deployment and installation • Software and documentation repository • Grid monitoring and problem tracking • Bug reporting and knowledge database • VO services • Grid management services NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 17

18. EGEE Networking Activity • Dissemination and outreach • User training and induction • Application identification and support • Two pilot application centers (for high energy physics and biomedical grids) • One more generic component dealing with longer term recruitment and support of other communities • Policy and International cooperation rely on a supporting network in the partner regions NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 18

19. EGEE Middleware Activity • Hardening and re-engineering of existing middleware functionality, leveraging the experience of partners • Activity concentrated in few major centers • Key services: Resource Access • Data Management • Information Collection and Accounting • Resource Brokering (Italy) • Quality Assurance • Grid Security • Middleware Integration • Middleware Testing NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 19

20. EGEE & Industry • Industrial participation encouraged both as potential end-users and IT technology and service suppliers • Normally through national and regional Grid EGEE federations • EGEE will maintain an Industry Forum to keep selected Industrial and Commercial interested parties in close contact • Services developed in first EGEE 2 years phase (2004-5) might be tendered to Industry in second phase (2006-7) NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 20

21. EGEE Timeline • May 2003: proposal submitted • July 2003: proposal accepted • September 2003: start negotiation • April 2004: start project NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 21

22. Conclusions • The EU DataGrid project has successfully fulfilled its role of EU Grid flagship project in collaboration with several other EU and international projects • Essential to keep the momentum and the current lead in production Grids in Europe • Important to build an international cooperation between European and US/AP Grid infrastructure projects • The scientific user communities are already international (HEP is an excellent example) and so the computing resources and most of the experimental instruments • EGEE proposes the right framework and plans to accomplish the above objectives leveraging present and future international research networks NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 22

23. More information • More information on the EU DataGrid project on: www.edg.org • More information on EGEE on: www.cern.ch/egee Or mail me at: fabrizio.gagliardi@cern.ch NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 23