1 / 14

Grid Metaschedulers

Grid Metaschedulers. An Overview and Up-to-date Solutions Christian Grimme christian.grimme@udo.edu. Outline. Grid Metaschedulers and Grid Brokers A unified view on Grid Scheduling Architecture Common functions of Grid Scheduling Up-to-date Grid Scheduling Solutions

kimberlygjones
Download Presentation

Grid Metaschedulers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Metaschedulers An Overview and Up-to-date Solutions Christian Grimme christian.grimme@udo.edu

  2. Outline • Grid Metaschedulers and Grid Brokers • A unified view on Grid Scheduling Architecture • Common functions of Grid Scheduling • Up-to-date Grid Scheduling Solutions • CSF (Community Scheduler Framework) • GridWay • Grid Service Broker (in GridBus) • GRMS (GridLab Resource Management System) • Others (Moab/Maui, WMS, Nimrod/G, Condor-G) Grid Metascheduler

  3. Grid Metaschedulers and Grid Brokers • General Problem: Allocation and scheduling of applications in a heterogeneous and dynamically changing set of resources • Three Scenarios: • Enterprise Grid • resources owned by single provider • central broker which interacts with local resource managers • HPC Grid • cooperation of HPC sites to maximize computational power • distributed administrative control, hierarchical brokering • Global Grid • all kind of resources • a broker accepts all kinds of jobs to forward them to other brokers or local resource schedulers Grid Metascheduler

  4. Common Functions of Grid Scheduling • Information • Discover and match resources for applications • Monitor internal strategies, feedback to user • Job Management • Submission of jobs and workflows • Scheduling functionality, forecasting, co-allocation • Reservation management • Security • Rights delegation • Access control • Agreement and Accounting • Service Level Agreements • Data access management Grid Metascheduler

  5. CSF (Community Scheduler Framework) • Open source project as part of the Globus project in cooperation with Platform Computing • Supports implementation of Grid metaschedulers based on GT services (MDS, GRAM, RFT, etc.) • Properties: • Enables communication between heterogeneous schedulers on local level • Allows reservations on resources • Offers simple scheduling mechanisms • Extensible via scheduling plug-ins Grid Metascheduler

  6. CSF Architecture Loads queue parameters and associated schedulers MetaScheduler Queuing Service Scheduling Plugin Info Service Reservation Service Job Service User Request Create and administer compute jobs Specified in RSL Reserve a resource if supported by LRM GRAM PBS GRAM SGE RM Adapter Binding to LRM PBS SGE LSF Grid Metascheduler

  7. GridWay • Open source component (Globus Proto Project) • Lightweight metascheduler on top of GT 2.4 – 4.x • Properties: • Support of GGF DRMA standard API for job submission and management • Simple scheduling mechanisms but extensible • Allows job dependencies (workflow) • Supports job migration/adaptive execution (Grid- and application-initiated) Grid Metascheduler

  8. GridWay Architecture RFT GRAM MDS DRMAA Library CLI Job control operations GridWay Core Request Manager Job pool Host pool Scheduler Dispatch Manager Matchmaking, execution and migration Transfer Manager Execution Manager Information Manager RFT GRAM MDS Execution of jobs on LRM Resource Grid Metascheduler

  9. GSB – Grid Service Broker (GridBus) • Metascheduler to access computational and data Grids • On top of various middlewares (GT 2.4 – 4.x, Alchemi, Unicore) • Properties: • Collaborates with Grid Economy, Trading and Accounting Services • Interacts with Workflow Management Service • Leverages economic scheduling model Grid Metascheduler

  10. GSB Architecture Analysis of job parameters, matchmaking Grid Economy Services Data Service Catalogue Parameter Resolver Resource Discovery Resource Catalogue Workflow Manage- ment Bookkeeper Grid Scheduler • Application Data • Parameters • Tasks Middleware binding for local job execution Grid Info Service Actuator & Monitor Simulation Mapping of jobs to resources Remote Data Host Agent Local Data Wraps and controls job actions (even transfer of distant data) Grid Metascheduler

  11. GRMS - GridLab Resource Management System • Open Source Metascheduler as part of the GridLab project • Based on GT 2.4 (assured to work on GT 4.x) • Properties: • Job dependencies are planned to be supported • Defines a Job Definition Language (GJD) • Simple scheduling mechanisms, extensible • Allows job migration • Provides a GSI enabled Web Service interface Grid Metascheduler

  12. GRMS Architecture Scheduling and matchmaking Jobs in GJD (with dependencies) GSI enabled Web Service Interface Other GridLab Services Workflow Module Matchmaking & Scheduler Plug-ins Replica Management Broker Module Job Queue File Movement Queuing of jobs ... Job Registry Resource Discovery Job Manager Middleware Service Layer GRAM Monitoring GridFTP Grid Metascheduler

  13. Others • Moab/Maui: • commercial Metascheduler with advanced scheduling an monitoring capabilities • on top of most major scheduling systems • Nimrod/G: • Grid aware solution of Nimrod for compute intensive tasks • works with GT, Legion, Condor • WMS: • no metascheduler, submits only to gLite CEs • has some interesting architecture features (push/pull models) • Condor-G: • simple queue submission system • treats a Grid as local resource Grid Metascheduler

  14. More information ... • Literature: • Tonellotto, N., Yahyapour, R., Wieder, Ph., A Proposal for a Generic Grid Scheduling Architecture, CoreGRID Technical Report 0015, Institute on Resource Management, 2006 • Buyya, R., Venugopal, S., A Gentle Introduction to Grid Computing and Technologies, CSI Communications, 2005 • Web Links: • CSF: www.globus.org/toolkit/docs/4.0/contributions/csf/ • GridWay: www.gridway.org • GSB: www.gridbus.org • GRMS: www.gridlab.org • Moab/Maui: www.clusterresources.com/pages/products/moab-grid-suite.php • WMS: http://egee-jra1-wm.mi.infn.it/egee-jra1-wm/wms.shtml • Nimrod/G: www.csse.monash.edu.au/~davida/nimrod/index.htm • Condor-G: www.cs.wisc.edu/condor/ Grid Metascheduler

More Related