1 / 18

Computational Grids and Computational Economy: Nimrod/G Approach

Computational Grids and Computational Economy: Nimrod/G Approach. David Abramson Rajkumar Buyya Jonathan Giddy. Coarse-grained SPMD model Execute one application repeatedly for many combinations of input parameters Legacy applications: add iteration and distribution without modifying code

harsha
Download Presentation

Computational Grids and Computational Economy: Nimrod/G Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computational Grids and Computational Economy: Nimrod/G Approach David Abramson Rajkumar Buyya Jonathan Giddy

  2. Coarse-grained SPMD model Execute one application repeatedly for many combinations of input parameters Legacy applications: add iteration and distribution without modifying code New applications: remove iteration and distribution from design Parametrised modeling experiments: Require very high levels of performance Generate Large amounts of work & concurrency Uncoupled computations Tolerate - moderately high latencies Parametric Execution of Applications

  3. Job 1 Job 2 Job 3 Job 4 Job 5 Job 6 Job 7 Job 8 Job 9 Job 10 Job 11 Job 12 Job 13 Job 14 Job 15 Job 16 Job 17 Job 18 Description of Parameters

  4. Nimrod (1994 - ) DSTC funded project Designed for department level clusters Proof of concept Clustor (Activetools) (1997 - ) Commercial version of Nimrod Re-engineered Features Workstation orientation Access to idle workstations Random allocation policy Password security Working with Small Clusters

  5. Manual resource location static file of machine names No resource scheduling first come first served No cost model all machines cost alike Single access mechanism Clustor limitations

  6. Towards Grid Computing…. Source: www.globus.org & updated

  7. Wide-Area Network Support redesigned architecture use of high-performance networks Scalable Scheduling “guaranteed” deadline use of existing schedulers Computational Economy “I am willing to pay $$, can you complete the job by given deadline” trading, bidding, resource reservation... Nimrod/G - Nimrod over Globus/Grid

  8. Layered Architecture (Grid Components) Applications High-level Services and Tools GlobusView Testbed Status Nimrod/G DUROC MPI MPI-IO CC++ globusrun Core Services Nexus GRAM Metacomputing Directory Service Globus Security Interface Heartbeat Monitor Gloperf GASS Local Services Condor MPI TCP UDP LSF Easy NQE AIX Irix Solaris Source: www.globus.org

  9. Nimrod/G Architecture Nimrod/G Client Nimrod/G Client Nimrod/G Client Parametric Engine Schedule Advisor Resource Discovery Persistent Info. Dispatcher Grid Directory Services Grid Middleware Services GUSTO Test Bed

  10. User process GASS server File access Nimrod/G Interactions • Additional services used implicitly: • GSI (authentication & authorization) • Nexus (communication) Resource location MDS server Scheduler Resource allocation (local) Prmtc.. Engine Dispatcher Queuing System Job Wrapper GRAM server Root node Computational node Gatekeeper node

  11. Find a set of machines (MDS search) Distribute jobs from root to machines Establish job consumption rate for each machine For each machine Can we meet deadline? If not, then return some jobs to root If yes, distribute more jobs to resource If cannot meet deadline with current resource Find additional resources Scheduling Algorithm

  12. Deadline Cost Available Machines A Nimrod/G Client

  13. Bioinformatics: Protein Modeling Sensitivity experiments on smog formation Parametric study of Laser detuning Combinatorial Optimization: Simulated Annealing Ecological Modeling: Control Strategies for Cattle Tick Electronic CAD: Field Programmable Gate Arrays Computer Graphics: Ray Tracing High Energy Physics: Searching for Rare Events Physics: Laser-Atom Collisions VLSI Design: SPICE Simulations Radiation Protection and Nuclear Safety Sample Applications of Nimrod

  14. Electronic CAD

  15. Some early results -

  16. AppLeS (UC. San Diego) application level scheduling & case-by-case NetSolve (UTK/ORNL) API for creating farms DISCWorld (U. Adelaide) remote information access Millennium (UC. Berkeley) remote execution environment on clusters and supports computational economy Related Works

  17. Nimrod/G architecture offers a scalable model for resource management and scheduling on computational grids Supports Computational Economy The current model supporting Parametric Computing can be extended to support parallel jobs or any other computational model. Plan to use the concept of Advance Resource Reservation in order to offer the feature wherein the user can say “I am willing to pay $…, can you complete my job by this time…” Further Information: www.csse.monash.edu.au/~davida/nimrod.html Conclusions

More Related