1 / 42

Sustainable EGI Infrastructure: 2020 Vision and Beyond

Explore EGI's efforts towards sustainability in research communities with digital research infrastructure, advancing healthcare, life sciences, and more. Learn about EGI's vision for 2020 and plans for Horizon 2020 funding.

bernal
Download Presentation

Sustainable EGI Infrastructure: 2020 Vision and Beyond

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building a sustainable EGI Steven Newhouse EGI-InSPIRE Project Director, EGI.eu Overview - EGI-InSPIRE Review 2013

  2. Contents • Digital Research Infrastructure in 2020 • Open Computing and Data Infrastructures • EGI-InSPIRE: An overview of PY3 • Moving towards service sustainability • A sustainable EGI • Governance, technical and financial aspects • Future Plans • For PY4 and beyond to funding in Horizon 2020 Overview - EGI-InSPIRE Review 2013

  3. Contents • Digital Research Infrastructure in 2020 • Open Computing and Data Infrastructures • EGI-InSPIRE: An overview of PY3 • Moving towards service sustainability • A sustainable EGI • Governance, technical and financial aspects • Future Plans • For PY4 and beyond to funding in Horizon 2020 Overview - EGI-InSPIRE Review 2013

  4. Research Communities Extracting Knowledge from the Data Deluge Requires a digital research infrastructure to share services and tools personalised to individual research communities • High-Energy Physics • Large Hadron Collider: World’s largest particle accelerator • Supports 8,000 researchers • 1 billion CPU hours in the last 12 months • 15PB of data created annually • Health-care • Diagnostic tools for the medical community • E.g.: to diagnose Alzheimer’s disease in its early stages and track the progress of the symptoms over time • Life Science • Structural Biology: provide data analysis • Structural Modeling: tools for NMR and SAXS techniques See: http://www.egi.eu/case-studies/ Overview - EGI-InSPIRE Review 2013

  5. Research Lifecycle Research Communities Research/e- Infrastructures Overview - EGI-InSPIRE Review 2013

  6. Digital Research Infrastructure • Digital Data Deluge • Instruments, Simulations, Archives, … • Support Researchers • Researchers from all disciplines • Research collaborations of all sizes • Researchers with different levels of expertise • Operational Infrastructure • Customised Virtual Research Environments • Diverse services for diverse researchers • Using open compute and data resources • Ensure every researcher is digital Overview - EGI-InSPIRE Review 2013

  7. EGI’s Vision for 2020 To support the digital European Research Area through a pan-European research infrastructure based on an open federation of reliable services that provide uniform access to national computing, storage and data resources. EGI 2020 Strategy: http://go.egi.eu/EGI2020 Overview - EGI-InSPIRE Review 2013

  8. Contents • Digital Research Infrastructure in 2020 • Open Computing and Data Infrastructures • EGI-InSPIRE: An overview of PY3 • Moving towards service sustainability • A sustainable EGI • Governance, technical and financial aspects • Future Plans • For PY4 and beyond to funding in Horizon 2020 Overview - EGI-InSPIRE Review 2013

  9. Response to Review (1/2) • DoW Amendments • Changes presented last year implemented • New WP8 (SA4) starting March 2013 defined • Effort from PY1 & PY2 to support SA4 in PY4 • Digital Research Infrastructure ERIC • Described later in these slides • Strategic Planning and Business Modeling • Service portfolio for EGI.eu Global Tasks • Prioritisation and cost established for portfolio • Baseline for a 1M Core and 1EB Storage Clouds Overview - EGI-InSPIRE Review 2013

  10. Response to Review (2/2) • Expand Project View • Clear focus on introducing cloud technologies • Peer review of pooled resources for excellent science • Assessing pay-per-use issues • Work with PRACE, EUDAT, HelixNebula & XSEDE • Outreach to new user communities • Shift from Heavy User Communities to next tier • Connect and develop using human networks • Develop and define the EGI Solutions Portfolio • Governance model remains the same Overview - EGI-InSPIRE Review 2013

  11. EGI-InSPIRE PY3 • Outputs • Virtual Teams • Funded Mini-projects • Technical and Community Forums • Evolving EGI Workshop • Demonstrating Excellent European Science • Digital Research ERIC • EGI/EUDAT/PRACE Use Cases • Developing the operational infrastructure • Defining our platform architecture • Integrating the federated cloud • Reinforcing our human networks • Recognising NGI Operations Managers • Consolidating NGI International Liaisons • Establishing EGI Champions • Planning around the EGI Geek Squad Overview - EGI-InSPIRE Review 2013

  12. EGI’s Human Networks See: Communication, Marketing & Events presentation this afternoon • Identifying the individuals in the Network • By both position and selection • Communication within the Network • Broadcast material for consumption and/or relaying • Social networking and discussion forums • Webinars to support development & training • Using the Network • Supporting EGI at meetings & events • Provide travel, registration & subsistence support • Direct Funding: All EGI Champions, some NGI Operations Managers • EGI-InSPIRE: NILs and NGI Operations Managers

  13. EGI Champions http://go.egi.eu/champs Afonso Duarte Biophysics & Structural biology FotisPsomopoulos Bioinformatics & Computer science Silvio Pardi Mathematics & Computer science Joeri van Leeuwen Astronomy Stella Arnaouti Civil engineering AshiqAnjum Biomedical sciences & Security EleniKatragkou Climatology & Climate science Mark Santcroos Biomedical sciences TomášKulhánek Computational physiology Overview - EGI-InSPIRE Review 2013

  14. Exploiting our Expertise • Human networks provide domain experts • Operations, User Support, Technology • Policy, Communications • Used to tackle community issues • New capabilities, new applications, … • Assemble into small focused teams • Virtual Teams: Unfunded investigative reports • Mini-Projects: Funded developments See: NA2 presentations this afternoon + SA1/JRA1 tomorrow Overview - EGI-InSPIRE Review 2013

  15. VTs: Bring Experts Together • Reports with recommendations • Scientific Gateway Primer • Scientific Discipline Classification • Scientific Publications Repository • New Capabilities • GPGPU Requirements within EGI • Inter NGI Usage reporting • New Communities • ELIXIR and EGI • Cherenkov Telescope Array Overview - EGI-InSPIRE Review 2013

  16. Mini-Projects • VTs restricted in resources • Existing EGI-InSPIRE staff • Un-funded staff in partners • Mini-projects provide dedicated resource • Open call within partners for small projects • Accelerating EGI’s Strategic Objectives • 6-12 months and costing ~€70K • Review and selection by the PMB • Established as WP8 (SA4) starting March 2013 • Explore as more agile and responsive model Overview - EGI-InSPIRE Review 2013

  17. Funded Mini-Projects • Virtual Research Environments • Massive Open Online Course Development • Evaluation of LifeRay Modules • Building and Using Operational Cloud Infrastructure • OCCI support for arbitrary Cloud Management Frameworks • CDMI Support in Cloud Management Frameworks • Dynamic Deployment for OCCI Compliant Clouds • Automatic Deployment and Execution of Applications Using Clouds • Transforming Research Platforms to Exploit Cloud Capabilities • Operational Tools • VO Administration Portal • A new approach to computing Availability and Reliability Reports • GOCDB Scoping Extensions and Management Interface • Automating applying for and allocating federated resources Overview - EGI-InSPIRE Review 2013

  18. Operational Infrastructure Registered Users: 21714 VOs: 233 LCPUs: 470,000 Disk: 143PB Tape: 138PB Jobs: 1.62 million/day Integrated EGI-InSPIRE Partners and EGI Council Members External Resource Providers (integrated) Internal/External Resource Providers (being integrated) Peer Resource Providers UPDATE Overview - EGI-InSPIRE Review 2013

  19. The EGI Platform Model • What is a platform? • “offers the developer an undertaking that code will run consistently” • Built upon a defined set of services/APIs/… • EGI needs platforms to provide a focus: • What it offers (i.e. capability or solution) • Who it is offered too (i.e. consumer) • How it is delivered (i.e. service portfolio) See: Overview of EGI’s Solution Portfolio in the next session Overview - EGI-InSPIRE Review 2013

  20. EGI’s Platforms for H2020 ERA needs an open ecosystem of VREs & services EGI Collaboration Platform “Tools & Services enabling cross-community collaboration” Community Platform (VRE) Community Platform (VRE) Community Platform (VRE) EGI Cloud Infrastructure Platform “A federated IaaS Cloud infrastructure” EGI Core Infrastructure Platform “Operational services necessary for the management of federated DCIs” Overview - EGI-InSPIRE Review 2013

  21. EGI’s Core Infrastructure The services that federate and integrate the functional services deployed in the production infrastructure EGI Core Infrastructure Platform Information Discovery (BDII) Federated AAI Monitoring Accounting Operations Portal Metrics Visualisation (gstat) Service Catalogue (GOCDB) Messaging For e-Infrastructures, Research Infrastructures & Community Platforms See: SA1 Presentation tomorrow

  22. EGI’s Cloud Infrastructure Enable an open ecosystem of services EGI Collaboration Platform EGI Cloud Infrastructure Platform VM Mgmt Storage Mgmt Information Image Repository Cloud Management Stacks (OpenStack, OpenNebula, …) Image Metadata Marketplace EGI Core Infrastructure Platform Federated AAI Central Service Registry Monitoring Accounting Overview - EGI-InSPIRE Review 2013 See: SA2 Presentation on Wednesday

  23. Community Platforms (CPs) • Can build on the Core Infrastructure • Deployed directly at cooperating sites • Limited number due to local effort required • Cloud Platform is a CP assembled by EGI • Can use the Cloud Infrastructure • Deploy on any Cloud enabled site • Effort from the deployerand not the site • CPs assembled by anyone Overview - EGI-InSPIRE Review 2013

  24. EGI’s Collaboration Platform • Shared services for all communities • People/Social Services • Technical/Infrastructure Services • Customisable VRE encompasses: • Services to access local resources • Community Platform deployed across a community • Services run for a single community • Community Platform with a few instances • Shared Services run for all communities • From the EGI Collaboration Platform See: Technical Outreach Presentation this afternoon Overview - EGI-InSPIRE Review 2013

  25. Contents • Digital Research Infrastructure in 2020 • Open Computing and Data Infrastructures • EGI-InSPIRE: An overview of PY3 • Moving towards service sustainability • A sustainable EGI • Governance, technical and financial aspects • Future Plans • For PY4 and beyond to funding in Horizon 2020 Overview - EGI-InSPIRE Review 2013

  26. An EGI ERIC? • EGI.eu becoming an ERIC • Possible value in some areas: tax and profile • Significant effort and risk for little reward • An ERIC for Digital Research Infrastructure • Bring together e-Infrastructure providers • Operate common and domain specific services • Consumers defining their service requirements • National representatives fund and govern Overview - EGI-InSPIRE Review 2013

  27. DRI ERIC Impact READ WHITE PAPAER • Provoked a lot of discussion at e-IRG • Representation of research communities • Responsibilities of operation vs. innovation • Sustainability of all e-Infrastructures • GEANT/DANTE/TERENA: Reykjavik group • Future plans of EUDAT, HelixNebula, … • Consequence • Discussing an ‘umbrella’ organisation/activity • Integration to provide single ERA interface • Retain independent operational units Overview - EGI-InSPIRE Review 2013

  28. Sustaining EGI Services Service Portfolio Activity within EGI Service Providers Resource infrastructure Providers Resource infrastructure Providers Resource infrastructure Providers Resource infrastructure Providers EGI.eu Technologists Researchers Overview - EGI-InSPIRE Review 2013

  29. EGI Service Portfolio • EGI.eu Service Portfolio • Services provided ‘centrally’ for all • Operated by EGI.eu and partners • Technical & human coordination services • Resource infrastructure Provider Portfolio • Technical & human coordination services • Ensures services from Resource Centres • Resource Centres provide EGI’s resources Overview - EGI-InSPIRE Review 2013

  30. EGI.eu Service Sustainability • Technical definition in the service portfolio • Critical: EGI can no longer function • Anti-Degradation: EGI degrades over time • Growth: Used to grow EGI • Minimal Impact: Service provided externally to EGI • Required financial support • Requirements, innovation and development • Coordination, routine operation and maintenance • Different activity will have different funding models • Reviewed at ‘Evolving EGI Workshop’ Jan 2013 Overview - EGI-InSPIRE Review 2013

  31. Contents • Digital Research Infrastructure in 2020 • Open Computing and Data Infrastructures • EGI-InSPIRE: An overview of PY3 • Moving towards service sustainability • A sustainable EGI • Governance, technical and financial aspects • Future Plans • For PY4 and beyond to funding in Horizon 2020 Overview - EGI-InSPIRE Review 2013

  32. Environment Social Sciences virtual research environments new professions & skills EIROs ESFRI CommunityData Services Centres ofExcellence research infrastructures Combustion Atmosphere/Space Physics industry big data management,preservation, computing PRACE RDA CompetenceCentres OpenAccess Astronomy … universities GEANT Biology Medicine DATA COMPUTING EGI Compute? Storage? Cloud? serviceintegration CONNECTIVITY From Kostas Glinos Dublin, May 2013 Overview - EGI-InSPIRE Review 2013

  33. Alignment with the EC Overview - EGI-InSPIRE Review 2013

  34. Tentative EC Timeline 2013 • June: Prepare draft work programmes • July: Meeting of Advisory Group (tentative) • Sept: Preparation of a consolidated draft • Nov: Consultation of Programme Committees • Dec 10th: Adoption by Commission • Dec 11th: Publication of calls for proposals Estimate projects start Autumn 2014 Overview - EGI-InSPIRE Review 2013

  35. Future EGI Priorities • Sustaining EGI’s production infrastructure • Governance, financialand technical issues • Providing Core & Basic Operational Services • Developing EGI’s human networks • Providing training, material & travel support • Engagement with research communities • Bring co-developed services into production • Developing EGI’s production infrastructure • Increasing scale, flexibility and scope Overview - EGI-InSPIRE Review 2013

  36. Sustaining EGI.eu Services • Service Consumer • Resource infrastructure Provider (NGIs) • Research Community • European Policy • Service Criticality • Critical: EGI can no longer function • Anti-Degradation: EGI degrades over time • Growth: Used to grow EGI • Minimal Impact: Service provided externally to EGI Overview - EGI-InSPIRE Review 2013

  37. After EGI-InSPIRE • Critical and anti-degradation Services • Supported by EGI.eu (Community Funds) • Rebidding underway driven by SLAs and costs • Growth and Support Services • Supported by EC project funds • Driven by priorities in Horizon 2020 • Result: A programme of co-funded activities • EGI.eu paying for routine operational activities • EC contributing to development and growth Overview - EGI-InSPIRE Review 2013

  38. EGI’s Grand Vision Establish by 2020 a distributed open compute and data infrastructure comprising a 1M Core Federated Cloud and 1 Exabyte of Federated Cloud Storage across Europe that is able to support the data analysis activities of all researchers within the European Research Area EGI Council Discussion – April 2013 Overview - EGI-InSPIRE Review 2013

  39. Within the Grand Vision… • European cloud to support excellent science • Federated through distributed national resources • National & regional support centres • Technical consultancy & training for researchers • Drive technical innovation in: • Tools to operate a uniform federated cloud • Services needed to build VREs • An easy to use platform for data analysis Overview - EGI-InSPIRE Review 2013

  40. Funding EGI Money Allows EGI to continue development and growth until Horizon 2020 starts Develop innovate new services and integrate into EGI to grow and diversify the supported research community Horizon 2020 Projects (EC Funds) 6 month extension EGI.eu fees EGI.eu fees ? EGI-InSPIRE: EC funds EGI-InSPIRE: EC funds Other EC Project Funds EGI.eu fees EGI-InSPIRE: Partner Contributions EGI-InSPIRE: Partner Contributions EGI-InSPIRE: EC funds Sustain EGI Services that are critical and prevent degradation EGI-InSPIRE: Partner Contributions EGI.eu fees EGI Community Funds EGI Participant Contributions May 2010 May 2013 Nov 2014 May 2014 Time Overview - EGI-InSPIRE Review 2013

  41. Over the last year • Defined the Core EGI Activities • And how they will be sustained through EGI.eu • Established human networks across EGI • Within the NGIs and researchers • New approaches to working and funding • Developing the Operational Infrastructure • Integration of different technologies (inc. cloud) • Engaging with new research communities Overview - EGI-InSPIRE Review 2013

  42. Plans for Next Year… • Implement stable and sustainable EGI • Baseline services supported by their consumers • Project based funding for innovation & development • Re-profile effort for PY4 + 6 month extension • Focus on activities for Horizon 2020 • Transition to new service and financing model • Complete and develop human networks • Complete the integration of Clouds into EGI • Continue increased automation to reduce costs Overview - EGI-InSPIRE Review 2013

More Related