1 / 28

Grid-Ireland

Grid-Ireland. John Morrison, University College Cork (UCC) Brian Coghlan , Trinity College Dublin (TCD) Andy Shearer, NUI Galway (NUIG) Ron Perrott, Queens University Belfast (QUB). Armagh 26-NOV-2002. Grid-Ireland. Grid-Ireland. Grid-Ireland Enterprise Ireland funded from Oct- 2000

thimba
Download Presentation

Grid-Ireland

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid-Ireland John Morrison, University College Cork (UCC) Brian Coghlan, Trinity College Dublin (TCD) Andy Shearer, NUI Galway (NUIG) Ron Perrott, Queens University Belfast (QUB) Armagh 26-NOV-2002 Grid-Ireland

  2. Grid-Ireland Grid-Ireland • Enterprise Ireland funded from Oct- 2000 • TCD, UCC, NUIG, QUB CosmoGrid: first virtual organization • HEA Funded from Oct-2002 • Dublin Institute for Advanced Studies with UCD, NUI, G, UCC, HEAnet, DCU, TCD, Met Office & Armagh Observatory • VO supported by Grid-Ireland & HEAnet

  3. CosmoGrid

  4. Grids What are they? coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations Ian Foster and Carl Kesselman (1998) Beowulf clusters and Grid computing technology will likely merge in the next decade … and together they form the Grid Gordon Bell and Jim Gray (Feb-2002)

  5. Grids: what are they ? • an architecture • IF a system acts as a unified resource • THEN we say it presents a single system image (SSI) • conceptual model = computer architecture • NOT client/server model • gives each user the illusion of a single machine • SSI components: • hardware / OS / services • SSI services: • login / user interface / admin / time / filesystem / …

  6. The Holy Grid Grail Available sometime in the next 15 years Submit a problem at a high level to the Grid The grid will ? • Analyse the problem: what code & data are available, where are they ? • Dynamically allocate tasks to available computational tasks • Determine the distribution and degree of parallelisation given the current grid resources • Security is automatic • Monitor remotely Concentrate upon the problem NOT how to compute it.

  7. Current Grid Architecture Space

  8. Application Developers System Managers Operating Systems Scientists CertificateAuthorities FileSystems UserAccounts BatchSystems PBS, LSF Storage Elements MassStorage Systems HPSS, Castor Computing Elements Grid Musicians

  9. Input Dataset UI JDL BLAST Output Dataset Job Submit Event Job Submission Service Computing Element StorageElement Grid Symphony Replica Catalog DB DB Resource Broker Information Service Job Status Logging & Bookkeeping

  10. CrossGrid DataGrid GLOBUS SITES Current Grid Architectures European CrossGrid [X#] • 11 countries / 21 instutions European DataGrid [EDG] • 11 countries / 21 instutions Globus • Argonne Labs + ISI Let’s look at these, starting with Globus

  11. Certification Authorities Cluster Nodes Gatekeeper Users Application Grid Information System NFS Server Disk Storage / MSS Network Monitoring Globus Execution Flows: single entry point

  12. Certification Authorities Gatekeepers (Cluster Nodes) Grid-Ireland gridkit 1.x works like this Users Application Grid Information System NFS Server Disk Storage / MSS Network Monitoring Globus Execution Flows: multiple entry points

  13. Applications Development Support MPICH-G Existing & Planned Grid Services GRAM Globus Replica Manager Replica Catalog GSI Globus-IO MDS GridFTP GASS GLOBUS SITES Physical Resources Notice how few services there are Local Filesystem Secondary Storage CPUs Grid-Ireland Architecture, 2001/2002

  14. Certification Authorities Worker Nodes Resource Broker Computing Element (Gatekeeper) Users Application VO Membership Grid Information System Storage Element Disk Storage / MSS VO Replica Catalog Network Monitoring DataGrid Execution Flows

  15. Applications Development Support MPICH-G Existing & Planned Grid Services Datagrid Broker RB DataGrid Replica Manager VO Management VOMS VO Authorization LCAS Grid Info System R-GMA Logging Bookkeeping L & B Grid Databases Spitfire DataGrid GRAM Globus Replica Manager Replica Catalog GSI Globus-IO MDS GridFTP GASS GLOBUS SITES Physical Resources Resource Manager SlashGrid Filesystem Resource Manager Local Filesystem Secondary Storage CPUs Grid-Ireland Architecture, 2002/2003

  16. Interactive Distributed Data Access Certification Authorities Grid Visualisation Kernel MPICH-G Grid Data Mining Portals Users Application VO Membership Distributed Data Collection MPI Verification Performance Analysis Roaming Access Metrics and Benchmarks GridMonitoring CrossGrid Execution Flows

  17. Applications Development Support MPICH-G MPI Verification Portals Performance Analysis Metrics and Benchmarks GridMonitoring Interactive Distributed Data Access Grid Visualisation Kernel Grid Data Mining Distributed Data Collection Roaming Access Existing & Planned Grid Services CrossGrid Datagrid Broker RB DataGrid Replica Manager VO Management VOMS VO Authorization LCAS Grid Info System R-GMA Logging Bookkeeping L & B Grid Databases Spitfire DataGrid GRAM Globus Replica Manager Replica Catalog GSI Globus-IO MDS GridFTP GASS GLOBUS Physical Resources Resource Manager Resource Manager Resource Manager SlashGrid Filesystem Resource Manager Resource Manager SITES Local Filesystem Secondary Storage CPUs Optimization of Data Access e.g. Visualization e.g. Instruments Tertiary Storage Grid-Ireland Architecture, 2003/2004

  18. Stress Testing Condensed Graphs Engine Existing Grid Middleware Internet + Hardware WebCom-G Project Core Contribution: Condensed Graphs Grid Middleware A NEW execution model for the Grid

  19. Execution Models • Execution can be described by an acyclic graph • Execution models may be classified by their FIRING RULE: • control driven  thread of control • data driven  opportunism • demand driven  laziness • Concepts: eager and lazy evaluation • Condensed graphs are a chameleon: • can execute a dynamic mix of all 3 models • can dynamically mix eager & lazy evaluation

  20. Operands arrive at the operand ports: • if in incorrect form, • are grafted into correct form • wait to be consumed • Firing rule requires operator & dest. too FIRE ! • Operator: • represents an appropriate process • consumes the operands • sends the result to destinations Condensed Graphs Firing Rule

  21. Certification Authorities Worker Nodes Resource Broker Computing Element (Gatekeeper) WebCom-G VO Membership Logging & Bookkeeping Grid Information System Storage Element Disk Storage / MSS VO Replica Catalog Network Monitoring WebCom-G: DataGrid Use-case Execution Flows

  22. Tightly Coupled Loosely Coupled Data Intensive Complex Compositions Stress Testing Condensed Graphs Grid Middleware WebCom-G Applications Development Support MPICH-G CrossGrid MPI Verification HDSM-G Metrics and Benchmarks Portals Performance Analysis DataGrid GridMonitoring Interactive Distributed Data Access Grid Visualisation Kernel Grid Data Mining Distributed Data Collection Roaming Access Existing & Planned Grid Services GLOBUS Datagrid Broker RB DataGrid Replica Manager VO Management VOMS VO Authorization LCAS Grid Info System R-GMA Logging Bookkeeping L & B Grid Databases Spitfire Resource Manager Resource Manager Resource Manager SlashGrid Filesystem Resource Manager Resource Manager SITES Secondary Storage Local Filesystem CPUs Optimization of Data Access e.g. Instruments e.g. Visualization Globus Replica Manager Replica Catalog GSI Globus-IO MDS GridFTP GASS GRAM Tertiary Storage Physical Resources Grid-Ireland WebCom-G Architecture, 2003/2004

  23. Certification Authorities Worker Nodes WebCom-G VO Membership Logging & Bookkeeping Grid Information System Storage Element Disk Storage / MSS VO Replica Catalog Network Monitoring WebCom-G: Pure CG Use-case Execution Flows

  24. Tightly Coupled Loosely Coupled Data Intensive Complex Compositions Stress Testing Condensed Graphs Grid Middleware WebCom-G Applications Development Support MPICH-G CrossGrid MPI Verification HDSM-G Metrics and Benchmarks Portals Performance Analysis DataGrid Interactive Distributed Data Access Grid Visualisation Kernel GridMonitoring Grid Data Mining Distributed Data Collection Roaming Access Existing & Planned Grid Services GLOBUS Datagrid Broker RB DataGrid Replica Manager Grid Info System R-GMA Grid Databases Spitfire GridFTP GASS Logging Bookkeeping L & B Resource Manager Resource Manager SlashGrid Filesystem Resource Manager Resource Manager Resource Manager SITES Local Filesystem Secondary Storage CPUs Optimization of Data Access e.g. Instruments e.g. Visualization VO Management VOMS VO Authorization LCAS GRAM GSI Globus-IO MDS Replica Catalog Globus Replica Manager Tertiary Storage Physical Resources Grid-Ireland WebCom-G Architecture, 2005/2006

  25. ‘in-the-loop’ Imaging for Surgery CosmoGrid Applications Mobile-Phone Electric Field Estimation LIGO/iVDGL Data Analysis Tightly Coupled Loosely Coupled Data Intensive Complex Compositions Stress Testing Condensed Graphs Grid Middleware WebCom-G Existing Grid Middleware Internet + Hardware WebCom-G: Stress Testing

  26. Grid-Ireland WebCom-G Grid Architecture Space Core Contribution: Condensed Graphs Grid Middleware

  27. Grids: 10 outstanding questions • Why don’t grids have more functionality yet ? • Why aren’t there more application developers and users ? • Where are the grid software development tools ? • Where are the grid system management tools ? • How do we make grids more secure ? • How do we define standard interfaces and definitions ? • How do we manage variance on the grid ? • How can the grid R & D be funded ? • What cost models are needed by the grid ? • What are the benefits to motivate sharing on the grid ? • THESE ARE JUST THE FIRST 10 QUESTIONS

  28. Grid-Ireland THE END

More Related