1 / 34

EGI: European Grid Infrastructure

EGI: European Grid Infrastructure. Steven Newhouse Interim EGI.eu Director EGI-InSPIRE Project Director Technical Director EGEE-III. European e-Infrastructure. European Data Grid (EDG) Explore concepts in a testbed Enabling Grid for E- sciencE (EGEE)

spiro
Download Presentation

EGI: European Grid Infrastructure

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EGI: European Grid Infrastructure Steven Newhouse Interim EGI.eu Director EGI-InSPIRE Project Director Technical Director EGEE-III

  2. European e-Infrastructure • European Data Grid (EDG) • Explore concepts in a testbed • Enabling Grid for E-sciencE (EGEE) • Moving from prototype to production • European Grid Infrastructure (EGI) • Routine usage of a sustainable e-infrastructure

  3. What is e-Infrastructure? • Resources linked by high speed networks • Compute, Storage, Instruments, ... • Controlled access to shared resources • Authentication, Authorisation, Accounting, ... • Dependable services for others to use • Driven by availability and reliability metrics • Services that are there for the long-term • Supporting experiments lasting decades Swiss Grid Day

  4. The EGEE-III Project 17,000 users 139,000 LCPUs (cores) 25Pb disk 39Pb tape 12 million jobs/month +45% in a year 268 sites +5% in a year 48 countries +10% in a year 162 Virtual Organisations +29% in a year Over 20 active communities in 112 VOs

  5. EGI Collaboration Research Community Research Community Research Community Research Community N G I Research Community N G I N G I E I R O N G I E I R O EGI.eu

  6. EGI.eu • Coordination for European Grid resources • Roadmap to integrate HTC, HPC, Data, Instruments, ... • Policy & services needed to run a grid • Governance & ownership by its stakeholders • EGI Council votes proportional to GDP • EGI Council fees proportional to votes • Sustainable small coordinating organisation (EGI.eu) • Builds on resources from within its stakeholders • Location selected to be Amsterdam • Distributed staff with a core (~50%) in Amsterdam • Approximately 40 staff, €3.5M/year Swiss Grid Day

  7. Relationship betweenNGIs & EGI EC Funding National Funding EGI.eu EGI Global Tasks (EC, NGI, EGI.eu) NGI National Tasks (NGI) NGI National Tasks (NGI) NGI National Tasks (NGI) NGI International Tasks (EC & NGI) NGI International Tasks (EC & NGI) NGI International Tasks (EC & NGI) Swiss Grid Day

  8. EGI means Innovation • Deploy Technology Innovation • Distributed Computing continues to evolve • Grids  Desktops  Virtualisation  Clouds ? • Enable Software Innovation • Provide reliable persistent technology platform • Today: Tools built on gLite/UNICORE/ARC • Support Research Innovation • Infrastructure for data driven research • Support for international research (e.g. ESFRI) Swiss Grid Day

  9. Technology Innovation • Will come from outside EGI • EGI runs DCI technologies in production • Partnership with technology projects • EMI (European Middleware Infrastructure) • Continued evolution of gLite, ARC, UNICORE • RESERVOIR • Public cloud functionality to ERA • StratusLab (Proposed) • Use RESERVOIR to support gLite environments Swiss Grid Day

  10. Innovating technology being deployed within the production infrastructure Technology innovations taking place outside the production infrastructure Collaboration • Release EC & National Research Projects NGIs & EIROs Feedback Swiss Grid Day

  11. Software Innovation • Will come from outside EGI • EGI is a neutral platform for applications • EGI cannot support all services in its core • Every community needs something different • Foster innovation within ‘sectors’ • High Throughput Computing • Digital Libraries • ... Swiss Grid Day

  12. Example of Deployed Services User Interface User Access Deployed by EGI Maintained & Supported by the community Deployed by EGI Maintained & Supported through EGI USER External Components RESPECT User Interface EGEE Maintained Components Information Services General Services Security Services Virtual Organisation Membership Service COMMUNITY SERVICES Workload Management Service Logging & Book keeping Service Hydra BDII Proxy Server AMGA File Transfer Service LHC File Catalogue Storage Element Compute Element CORE UMD SITE SERVICES (TBC) CREAM LCG-CE SCAS Disk Pool Manager Authz. Service BLAH MON LCAS & LCMAPS dCache Worker Node gLExec BDII Physical Resources

  13. Digital Library Community • Community specific software stack • gCube from D4Science project • VREs: Manage access to libraries • Big focus on usability Information Service Virtual Research Environment (VRE) Manager Service Container VRE Services VRE Services Core UMD Services Swiss Grid Day Physical Resources

  14. High Throughput Computing • Managing large collections of jobs & files • HEP, LS, ... • Problem specific Applications and Tools • Maintained & supported by that community GANGA DIANE WISDOM User Applications HTC Community Services (e.g. WMS, FTS, LFC , AMGA, ...) Core UMD Services Swiss Grid Day Physical Resources

  15. Research Innovation • An infrastructure to support the ERA • Within the EU27 • Geographical Europe • Interoperability worldwide for collaboration • Work with Virtual Research Communities • Groupings of aligned Virtual Organisations • Provide community specific: • Support, training, consultancy, requirements etc. Swiss Grid Day

  16. The ESFRI Projects • First roadmap 2006 and updated in 2008 with 44 projects • Preparatory phase funding for most with second round soon • Big push in FP8? ESFRI: European Strategy Forum on Research Infrastructures Swiss Grid Day

  17. ESFRI Environmental Sciences IAGOS-ERI EURO-ARGO SIOS Status 2009 EUFAR-COPAL AURORA BOREALIS LIFEWATCH EISCAT-3D EPOS EMSO Swiss Grid Day ICOS

  18. EGI User Support USERS EGI.eu EGI Helpdesk NGI VOs User Forum Steering Committee Virtual Research Community USERS Training Events VOs Trainers VOs Virtual Research Community Apps. DB USERS VOs VRC Helpdesk ESFRI Project Other Helpdesk Virtual Research Community VOs NGI Helpdesk

  19. Moving from EGEE to EGI • What is different? • EGEE did ‘everything’ • EGI focuses just on infrastructure operations • What is the same? • Running a 24/7 production quality infrastructure • Providing a support framework for the users • Collaboration to drive European DCI forward Swiss Grid Day

  20. The EGI-InSPIRE Project Integrated Sustainable Pan-European Infrastructure for Researchers in Europe • Submitted to INFRA-2010 1.2.1 • Targeting call objectives: • 1.2.1.1: European Grid Initiative • 1.2.1.2: Service deployment for heavy users • A 4 year project • Seeking a total €25M EC contribution • Project cost €70M

  21. How much does it cost? Annual Costs EGEE EC Contribution: €16M (EGI Equivalent: ~€11M) Infrastructure Cost: ~ €80M

  22. EGI Project Ecosystem Virtual Research Communities Community Supporting Projects VRC SSC Specialised Support Centres Community VRC SSC EGI VRC EC Projects Software Providers Software Providers Software Providers

  23. Project Structure External Relations: Policy, Dissemination and Standards Software Provisioning and Support Operations and Tools User Support and Services SA2 Middleware Unit JRA1 Operational Tools NA3 User Support SA3 Software Support SA4 Services for HUCs SA1 Operations Project Management

  24. Project Structure Technical Collaborations (e.g. CHAIN, GISELR, SIENA, ...) EGI-InSPIRE Software Projects (e.g. EMI, IGE, SGI...) VRC & SSC Projects (e.g. ROSCOE, SAFE, CUE, TAPAS, ...) Managerial Collaborations (e.g. EC, DEISA, OSG, ...)

  25. User Support & Services • Support User Communities • Researchers in International Collaborations • Scale up from the single VO to a community • Provide a federated Helpdesk linking: • Discipline specific support (e.g. Bio Apps) • National infrastructure support (e.g. NGS) • Generic services (e.g. Training) • Provide core services to support users • Manage VOs, Application DB, Training DB Swiss Grid Day

  26. Possible Virtual Research Communities (FP7 funding) • High Energy Physics (ROSCOE) • Life Sciences (ROSCOE) • Computational Chemistry & Material Science (ROSCOE) • Grid Observatory [Computer Science] (ROSCOE) • Complexity Science (ROSCOE) • Photon Science (ROSCOE) • Humanities (ROSCOE) • Astronomy & Astrophysics (SAFE) • Fusion (SAFE) • Earth Science (SAFE) • Molecular Biodiversity (SIMBIOME) • NMR in Structural Biology (WeNMR) • Hydro-Meteorology (DRIHME) Swiss Grid Day

  27. Training Events

  28. Registry of Trainers

  29. Applications Database http://appdb.eu-egee.org Alt. link: http://grid.ct.infn.it/egee_applications/

  30. EGEE RESPECT  EGI • Identify third-party software that works well with Grid • gLite: http://technical.eu-egee.org/index.php?id=290 • Expand to include UNICORE, ARC, Globus, … • Simplified Access • P-GRADE, Ganga, Migrating Desktop, g-Eclipse, i2glogin, Virtual Control Room • Workload Management • GridWayMetascheduler, DIANE • New Resources • GRelC, Instrument Element • Infrastructure Services • StoRM Swiss Grid Day

  31. Other Activities • Dissemination • With NGIs, VRCs, SSCs and other projects • Support for Heavy User Communities • General & community specific services • Events • Two Annual meetings: Users & Technology • Technology • Liaison with software providers • Definition and verification of requirements Swiss Grid Day

  32. Resources • EGI.eu does not own or control resources • Resources owned by individual organisations • They manage access for their user communities • Middleware enables secure authorised access for external collaborators • EGI provided Unified Middleware Distribution (UMD) • EGI defined software & interfaces • UMD compatible software must be deployed • Interoperation within your country and internationally Swiss Grid Day

  33. Transition to EGI Model • Has already started within EGEE-III • Will continue in EGI-InSPIRE • Integrated Sustainable Pan-European Infrastructure for Researchers in Europe • 4 year project, seeking €25M EC contribution • User facing changes will be gradual • Focused communication from EGI & EGEE • Main public points of contact have few changes • Helpdesk & software Swiss Grid Day

  34. Summary • Move away from a project based model • Sustainable e-Infrastructure • User services will evolve to this new model • Coordination of services provided through NGIs • Expect little change in the delivered services • Start coordinating European e-Infrastructure • 1st May 2010 • Contact: director@egi.eu Swiss Grid Day

More Related