1 / 17

EDGI Brings Desktop Grids To Distributed Computing Interoperability

EDGI Brings Desktop Grids To Distributed Computing Interoperability. Etienne URBAH urbah@lal.in2p3.fr OGF29 Chicago LAL, Univ Paris-Sud, IN2P3/CNRS, Orsay, France. Summary. EDGI project : Context

adila
Download Presentation

EDGI Brings Desktop Grids To Distributed Computing Interoperability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EDGI Brings Desktop Grids To Distributed Computing Interoperability Etienne URBAH urbah@lal.in2p3.fr OGF29 Chicago LAL, Univ Paris-Sud, IN2P3/CNRS, Orsay, France The EDGI project receives Community research funding

  2. Summary • EDGI project : Context • EGEE = Service Grid focused on Secure Data Sharing • DG = Desktop Grid = Loose opportunistic grid using idle resources • EDGeS project :Presentation, 3G Bridge, Infrastructure, Gflop/s • EDGI project : Goal = Grid + Cloud • Bridging Security and Info • Useful Official and De facto Standards • Interfaces : Diagrams – Scope of JSDL + BES Author : E. Urbah

  3. EDGI project : Context Achievements of EDGeS • Modular design of 3G Bridge with Application Repository, permitting integration between various DGs and SGs • 3G Bridges in full production for SGs implementing gLite : • BOINC and XWHEP <--> EGEE, • OurGrid <--> EELA2. • ATTIC = Proof of Concept of Data Distribution Cluster based service grids (SGs) (EGEE, NDGF, ...) Supercomputer based SGs (DEISA, ...) OGF PGI EDGeS EDGI Desktop grids (DGs) (Volunteer DGs – Home computers, Organizational DGs – Institutional desktops) EDGI Academic Clouds Author : E. Urbah

  4. EGEE = Service Grid focused on Secure Data Sharing VOMS Admin Site Admin VOMS Server Manages VO X509 proxy VOMS Extensions Manages Site Trust Anchor = IGTF Publishes the Root CA Certificates Interoperability with another Grid Infrastructure is possible only if it uses IGTF as Trust Anchor Grid User Site Storage Resource VOMS Proxy =X509 proxy with VOMS extensions Accesses Data with VOMS proxy Author : E. Urbah

  5. EGEE = Service Grid focused on Secure Data Sharing VOMS Admin Grid Admin Site Admin VOMS Server Manages VO AccountingLogging & Bookkeeping X509 proxy Gives Accounting and Auditing VOMS Extensions Manages Site Gives Job Status Log Log Publishes available Resources Submits Job with VOMS proxy Meta-scheduler(WMS) Site Computing Resource Pushes Job Sends back Output Sandbox Sends back Output Sandbox Grid User Accesses Data with VOMS proxy Site Storage Resource VOMS Proxy =X509 proxy with VOMS extensions Accesses Data with VOMS proxy Author : E. Urbah

  6. DG = Desktop Grid = Loose opportunistic grid using idle resources WU = Work Unit = Application + Input Data Resource Owner(often volunteer) Currently, for BOINC, both roles of ‘Application Manager’ and ‘Grid User’ are fulfilled by ‘BOINC Project Owners’. Application Manager Accepts or Refuses an application on his resource Certifies Application Owns Resource Requests WU Submits input data for an application Grid Server withApplication Repository Computing Resource (often Desktop Computer) Sends WU Sends back results Sends back results Grid User Author : E. Urbah

  7. EDGeS project : Presentation FP7 project started in 2008 and just ended • Integrate Service Grids and Desktop Grids • Enable very large number of computing resources (100K-1M processors) • Attract new scientific communities • Provide a Grid application development environment • Provide application repository and bridges for the execution in the SG-DG system WLCG (CERN) gLite (EGEE, EELA, GridPP) VDT (OSG) ARC (NDGF) Unicore (DEISA) Future Prototype with BES Operational EDGeS Future GT4 (NGS) Operational Boinc (Berkeley) XWHEP (INRIA/IN2P3) OurGrid (Brazil) Author : E. Urbah

  8. EDGeS 3G Bridge : Modular Design Control path WU  Job EDGeS 3G Bridge NDGF Work Unit DEISA JobDatabase User BES Plugins Job with VOMS proxy BOINC Server BOINC Handlers EGEE WU EGEE Plugins gLite WMS EGEE Job Queue Manager Source Grid Handler Interface Target Grid Plugin Interface Control path Job  WU gLite WMS BOINC Plugins (DC-API) Scheduler EGEE Handler Job BOINC Server LCG-CE for EDGeS WU XWHEP Plugins XWHEP Server Author : E. Urbah

  9. EDGeS Infrastructure : Principle of Operation Author : E. Urbah

  10. EDGeS operational EGEE  DG infrastructure EDGeS VO of EGEE EDGeS BOINC-based Desktop Grids SZDG (public) 80.000 PCs UoW (local) 3.100 PCs SZTAKI CE 16 cpus BOINC  EGEE bridge IberCivis (public) 28.000 PCs EDGeS@ home (public) 3.300 PCs CNRS / IN2P3 CE 1.600 cpus EGEE  BOINC bridge AlmereGrid (public) 1.800 PCs CIEMAT CE 20 cpus Application Repository BOINC Project Owners BDII VOMS XWHEP-based Desktop Grids EGEE  XWHEP bridge WMS LB LAL (local) 80 PCs IN2P3 (public) 30 PCs MyProxy AlmereGrid (local) 10 PCs XWHEP Agent XWHEP Users EGEE Users Author : E. Urbah

  11. EDGeS@Home – GFLOP/s (7 days) Author : E. Urbah

  12. EDGI project : Goals • On the model of IberCivis, creation of a self-sustainable EuroCivis • Integration between DGs and SGs implementing BES (ARC, Unicore) • Bridge to Academic Clouds for Jobs needing QoS • Long term support of the EDGI bridges by the NGIs • Secure Data Distribution using ATTIC Cluster based service grids (SGs) (EGEE, NDGF, ...) Supercomputer based SGs (DEISA, ...) OGF PGI EDGeS EDGI Desktop grids (DGs) (Volunteer DGs – Home computers, Organizational DGs – Institutional desktops) EDGI Academic Clouds Author : E. Urbah

  13. BOINC  EGEE Bridging Security EDGeS 3G bridge EGEE BOINC Handler1 for each (BOINC server, BOINC Project Owner, EGEE VO) triple Queue Manager & Job DB EGEE Plugin1 for each (BOINC Project Owner, EGEE VO) pair WMS Work Unit BOINC jobwrapper client (simulating a large BOINC computing resource) WUi+1 Job Source Grid Handler Interface BOINC Server Target Grid Plugin Interface Jobi+1 WUi+2 3G job-wrapper 3G job-wrapper Jobi+2 WUi+3 VOMS Server Submission DN of X509 proxy Config. file VOMS proxy Retriever VOMS extensions Short term X509 proxy MyProxy trusting EDGeS 3G bridge Medium term X509 proxy BOINC Project Owner Author : E. Urbah

  14. EGEE  BOINC Bridging Info EGEE BOINC Server gLite WMS EGEE BDII LCG-CE for EDGeS EDGeS3G bridge Informationprovider Reports resources and performance Gets Infos Queue Manager& Job DB Sends output Logs events Submits Job Pushes job GRAM JobManagerfor EDGeS EGEE LB Logs events Generic Job WS Handler Adds job Watches job Gets output EGEE VOMS Watches BOINC plugin (DC-API) Gets VOMS proxy Checks EXE EDGeS Application Repository BOINC Computing Resource Gets EXE BOINC Service EGEE User Author : E. Urbah

  15. Grid + Cloud : Useful Official and De facto Standards Author : E. Urbah

  16. Interfaces for Grid + Cloud : Abstract Diagram Author : E. Urbah

  17. Interfaces for Grid + Cloud : Detailed Diagram – Scope of JSDL + BES Author : E. Urbah

More Related