1 / 48

The Future of EGEE and gLite

The Future of EGEE and gLite. Dieter Kranzlmüller GUP – Institute of Graphics and Parallel Processing Joh. Kepler Univ. Linz, Austria. CE EGEE&SEEGRID-2 Summer School on Grid Appl. 08 July 2006 Budapest, Hungary.

titus
Download Presentation

The Future of EGEE and gLite

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Future of EGEE and gLite Dieter Kranzlmüller GUP – Institute of Graphics and Parallel ProcessingJoh. Kepler Univ. Linz, Austria CE EGEE&SEEGRID-2 Summer School on Grid Appl. 08 July 2006 Budapest, Hungary

  2. GUP – Institute of Graphics and Parallel ProcessingJoh. Kepler University Linz EGEE – Enabling Grids for E-sciencECERN, Geneva, Switzerland Austrian Grid Initiative GGF - Global Grid Forum e-IRG - e-Infrastructure Reflection Group The Future of EGEE and gLite

  3. Lost in Definitions? Defining the “Grid”: • Access to (high performance) computing power • Distributed parallel computing • Improved resource utilization through resource sharing • Increased memory provision • Controlled access to distributed memory • Interconnection of arbitrary resources (sensors, instruments, …) • Collaboration between users/resources • Higher abstraction layer above network services • Corresponding security • … The Future of EGEE and gLite

  4. Defining the Grid • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. • This interconnection of users, resources, and services for jointly addressing dedicated tasks is called a virtual organization. • Comparison between Grids and Networks: • Networks realize message exchange between endpoints • Grids realize services for the users  higher level of abstraction The Future of EGEE and gLite

  5. Defining the Grid • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. The Future of EGEE and gLite

  6. The EGEE Project • EGEE • 1 April 2004 – 31 March 2006 • 71 partners in 27 countries, federated in regional Grids • EGEE-II • 1 April 2006 – 31 March 2008 • Expanded consortium • 91 partners • 11 Joint Research Units (48 partners) • Exploitation of EGEE results • Emphasis on providing production-level infrastructure  increased support for applications  interoperation with other Grid infrastructures  more involvement from Industry The Future of EGEE and gLite

  7. Defining the Grid • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. Status of EGEE-II (as of July 8, 2006) The Future of EGEE and gLite

  8. EGEE Infrastructure Scale (June 2006): ~ 200 sites in 40 countries ~ 20 000 CPUs > 10 PB storage > 20 000 concurrent jobs per day > 60 Virtual Organizations Country participating in EGEE The Future of EGEE and gLite

  9. EGEE Infrastructures • Production service • Scaling up the infrastructure with resource centres around the globe • Stable, well-supported infrastructure, running only well-tested and reliable middleware • Pre-production service • Run in parallel with the production service (restricted nr of sites) • First deployment of new versions of the gLite middleware • Test-bed for applications and other external functionality • T-Infrastructure (Training&Education) • Complete suite of Grid elements and application (Testbed, CA, VO, monitoring, …) • Everyone can register and use GILDA for training and testing 15 sites on 3 continents(all of them GÉANT sites) The Future of EGEE and gLite

  10. Defining the Grid • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. Status of EGEE-II (as of July 8, 2006) The Future of EGEE and gLite

  11. Production Grid Middleware Key factors in EGEE Grid Middleware Development: • Strict software process Use industry standard software engineering methods • Software configuration management, version control, defect tracking, automatic build system, … • Conservative approach in what software to use Avoid “cutting-edge” software • Deployment on over 100 sites cannot assume a homogenous environment – middleware needs to work with many underlying software flavors Avoid evolving standards • Evolving standards change quickly (and sometime significantly cf. OGSI vs. WSRF) – impossible to keep pace on > 100 sites Long (and tedious) pathfrom prototypes to production The Future of EGEE and gLite

  12. LCG-2 gLite 2004 prototyping prototyping product 2005 product 2006 EGEE Middleware: gLite • Exploit experience & existing components • VDT (Condor, Globus) • EDG/LCG • AliEn • … • Develop a lightweight stack of EGEE generic middleware • Dynamic deployment • Pluggable components • Focus is on re-engineering and hardening • March 4, 2006: gLite 3.0 gLite 3.0 The Future of EGEE and gLite

  13. Developing • gLite 3.0 now available on production infrastructure • After gLite 3.0: • Continuous release of single components • As needed by users and as made available by developers • Major releases provide a “check-point” • In general in coincidence with major application challenges • Continuing development to • Bring components not yet included in release to maturity • Improve functionality • Increase robustness • Increase usability • Improve the compliance to international standards The Future of EGEE and gLite

  14. GIN Grid Interoperability Leading role in building world-wide grids • Incubator for new Gridprojects world-wide • Interoperation efforts • Bilateral: EGEE/OSG, EGEE/NDGF, EGEE/NAREGI • Multilateral: Grid Interoperability Now (GIN) • Experiences and requirements fed back into standardization process (GGF) • Strengthening contacts with industry The Future of EGEE and gLite

  15. Applications Environmental Sciences Life & Pharmaceutical Sciences Geo Sciences Middleware APST Globus GT4 Condor Building Software for the Grid Courtesy IBM Platform Infrastructure Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Slide Courtesy David Abramson The Future of EGEE and gLite

  16. Applications Environmental Sciences Life & Pharmaceutical Sciences Geo Sciences Middleware APST Globus GT4 Condor Building Software for the Grid Upper Middleware & Tools Lower Middleware Courtesy IBM, Bonds Platform Infrastructure Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Slide Courtesy David Abramson The Future of EGEE and gLite

  17. Defining the Grid • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. Status of EGEE-II (as of July 8, 2006) The Future of EGEE and gLite

  18. EGEE Applications • >20 applications • High Energy Physics • Biomedicine • Earth Sciences • Computational Chemistry • Astronomy • Geo-Physics • Financial Simulation • Fusion • Further applications in evaluation Applications now moving from testing to routine and daily usage The Future of EGEE and gLite

  19. Mont Blanc (4810 m) Downtown Geneva High Energy Physics Large Hadron Collider (LHC): • One of the most powerful instruments ever built to investigate matter • 4 Experiments: ALICE, ATLAS, CMS, LHCb • 27 km circumference tunnel • Due to start up in 2007 The Future of EGEE and gLite

  20. Applications Example: WISDOM • Grid-enabled drug discovery process for neglected diseases • In silico docking • compute probability that potential drugs dock with target protein • To speed up and reduce cost to develop new drugs • WISDOM (World-wide In Silico Docking On Malaria) • First biomedical data challenge • 46 million ligands docked in 6 weeks • Target proteins from malaria parasite • Molecular docking applications: Autodock and FlexX • ~1 million virtual ligands selected • 1TB of data produced • 1000 computers in 15 countries • Equivalent to 80 CPU years • Significant results • Best hits to be re-ranked using Molecular Dynamics New data challenge planned for autumn 2006 The Future of EGEE and gLite

  21. H5 N1 Credit: Y-T Wu Credit: Y-T Wu Example: Avian flu • Avian Flu H5N1 • H5 and N1 = proteins on virus surface • Biological goal of data challenge • Study in silico the impact of selected point mutations on the efficiency of existing drugs • Find new potential drugs • Data challenge parameters: • 5 Grid projects: Auvergrid, BioinfoGrid, EGEE, Embrace, TWGrid • 1 docking software: autodock • 8 conformations of the target (N1) • 300 000 selected compounds >100 CPU years to dock all configurations on all compounds • Timescale: • First contacts established: 1 March 2006 • Data Challenge kick-off: 1 April 2006 • Duration: 4 weeks The Future of EGEE and gLite

  22. Sumatra, March 28, 2005 Mw=8.5 Peru, June 23, 2001 Mw=8.4 Example: Determining earthquake mechanisms • Seismic software application determines epicentre, magnitude, mechanism • Analysis of Indonesian earthquake (28 March 2005) • Seismic data within 12 hours after the earthquake • Solution found within 30 hours after earthquake occurred • 10 times faster on the Grid than on local computers • Results • Not an aftershock of December 2004 earthquake • Different location (different part of fault line further south) • Different mechanism Rapid analysis of earthquakes important for relief efforts The Future of EGEE and gLite

  23. EGEE-II Overview The Future of EGEE and gLite

  24. EGEE and Sustainability BUT … • How does EGEE compare to other computing infrastructures? • Number of infrastructure users? • Number of application domains? • Number of computing nodes? • Number of years in service? • What would happen, if EGEE is turned off? • What happens after April 2008 (End of EGEE-II)? The Future of EGEE and gLite

  25. The Future of Grids • Increasing the number of infrastructure users by increasing awareness • Dissemination and outreach • Training and education • Increasing the number of applications by improving application support and middleware functionality • High level grid middleware extensions • Increasing the grid infrastructure • Incubating related projects • Ensuring interoperability between projects • Protecting user investments • Towards a sustainable grid infrastructure The Future of EGEE and gLite

  26. User Information & Support • More than 170 training events and summer schools across many countries • >3000 people trained induction; application developer; advanced; retreats • Material archive online with ~250 presentations • Public and technical websites • Dissemination material  constantly evolving to expand information and keep it up to date • 4 conferences organized (~ 460 @ Pisa) • Next conference: September 2006 in Geneva ~600 participants The Future of EGEE and gLite

  27. Industry and EGEE-II • Industry Task Force • Group of industry partners in the project • Links related industry projects (NESSI, BEinGRID, …) • Works with EGEE’s Technical Coordination Group • Collaboration with CERN openlab project • IT industry partnerships for hardware and software development • EGEE Business Associates (EBA) • Companies sponsoring work on joint-interest subjects • Industry Forum • Led by Industry to improve Grid take-up in Industry • Organises industry events and disseminates grid information The Future of EGEE and gLite

  28. The Future of Grids • Increasing the number of infrastructure users by increasing awareness • Dissemination and outreach • Training and education • Increasing the number of applications by improving application support and middleware functionality • High level grid middleware extensions • Increasing the grid infrastructure • Incubating related projects • Ensuring interoperability between projects • Protecting user investments • Towards a sustainable grid infrastructure The Future of EGEE and gLite

  29. Applications Environmental Sciences Life & Pharmaceutical Sciences Geo Sciences ??? Middleware APST Globus GT4 Condor Building Software for the Grid Upper Middleware & Tools Lower Middleware Courtesy IBM, Bonds Platform Infrastructure Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Slide Courtesy David Abramson The Future of EGEE and gLite

  30. Portals on EGEE P-Grade Genius The Future of EGEE and gLite

  31. www.austriangrid.at High Level Middleware Extensions Example: • Understanding data through graphical representations  Scientific Visualization • Approach: Grid Visualization Kernel (GVK) • Interactive visualization service in the grid • Integrated in existing visualization toolkits • Optimized transportation and processing • Research project: • Interactivity on the Grid  glogin • Batch-bypass technology • Easy to use/install • Secure The Future of EGEE and gLite

  32. Interactive Bidirectional Connection glogin - Interactive Tunneling Client Gatekeeper WorkerNode Point ofContact glogin WorkerNode WorkerNode glogin’ Traffic Forwarding socket WorkerNode WorkerNode on the Grid The Future of EGEE and gLite

  33. glogin Shell • Interactive access to grid nodes • Authentication via grid certificates • Tunneling of arbitary traffic The Future of EGEE and gLite

  34. inter-action vis.data vis.data vis.data interaction interaction Grid Visualization Kernel Workernode GridVisualization Kernel Workernode WORKING! Workernode glogin’ Client glogin Workernode on the Grid The Future of EGEE and gLite

  35. Example: Biomedicine • Parallel simulationof blood flowon the Grid • Onlinevisualizationof simulationresults on thedesktop • Interactivesteering ofsimulation • Grid is„invisible“ Cooperation with University Amsterdam The Future of EGEE and gLite

  36. Example: Flooding Crisis Support • Simulation of floodingon the Grid • Onlinevisualizationof simulationresults in theCAVE • Interactivesteering ofsimulation • Grid is„invisible“ Cooperation with Slowak Academy of Sciences The Future of EGEE and gLite

  37. Scientific Visualization Use your favourite device to connect to the Grid: Sony PSP – PlayStation Portable The Future of EGEE and gLite

  38. Applications Environmental Sciences Life & Pharmaceutical Sciences Geo Sciences GENIUS, P-Grade, GVK, glogin, GVid, … Middleware APST Globus GT4 Condor Building Software for the Grid Upper Middleware & Tools Lower Middleware Courtesy IBM, Bonds Platform Infrastructure Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Slide Courtesy David Abramson The Future of EGEE and gLite

  39. The Future of Grids • Increasing the number of infrastructure users by increasing awareness • Dissemination and outreach • Training and education • Increasing the number of applications by improving application support and middleware functionality • High level grid middleware extensions • Increasing the grid infrastructure • Incubating related projects • Ensuring interoperability between projects • Protecting user investments • Towards a sustainable grid infrastructure The Future of EGEE and gLite

  40. EU GRID Projects related to EGEE The Future of EGEE and gLite

  41. GIN Related Infrastructures The Future of EGEE and gLite

  42. The Future of Grids • Increasing the number of infrastructure users by increasing awareness • Dissemination and outreach • Training and education • Increasing the number of applications by improving application support and middleware functionality • High level grid middleware extensions • Increasing the grid infrastructure • Incubating related projects • Ensuring interoperability between projects • Protecting user investments • Towards a sustainable grid infrastructure The Future of EGEE and gLite

  43. Sustainability: Beyond EGEE-II • Need to prepare for permanent Grid infrastructure • Maintain Europe’s leading position in global science Grids • Ensure a reliable and adaptive support for all sciences • Independent of project funding cycles • Modelled on success of GÉANT • Infrastructure managed centrally in collaboration with national bodies (in EGEE-II: JRUs) The Future of EGEE and gLite

  44. Grids in Europe • Great investment in developing Grid technology • Sample of National Grid projects: • Austrian Grid Initiative • DutchGrid • France: Grid’5000 • Germany: D-Grid; Unicore • Greece: HellasGrid • Grid Ireland • Italy: INFNGrid; GRID.IT • NorduGrid • Swiss Grid • UK e-Science: National Grid Service; OMII; GridPP • EGEE provides framework for national, regional and thematic Grids The Future of EGEE and gLite

  45. EDG EGEE EGEE-II EGEE-III Routine Usage Testbeds Utility Service Evolution European e-InfrastructureCoordination The Future of EGEE and gLite

  46. Summary Grids represent a powerful new tool for science Today we have a window of opportunity to move grids from research prototypes to permanent production systems (as networks did a few years ago) EGEE offers … • … a mechanism for linking together people, resources and data of many scientific community • … a basic set of middleware for gridfying applications with documentation, training and support • … regular forums for linking with grid experts, other communities and industry The Future of EGEE and gLite

  47. Summary • Success will lead to the adoption of grids as the main computing infrastructure for science • If we succeed then the potential return to international scientific communities will be enormous and open the path for commercial and industrial applications The Future of EGEE and gLite

  48. EGEE’06 Conference • EGEE’06 – Capitalising on e-infrastructures • Demos • Related Projects • Industry • International community (UN organisations in Geneva etc.) • 25-29 September 2006 • Geneva, Switzerland http://www.eu-egee.org/egee06 The Future of EGEE and gLite

More Related