1 / 11

The GridWay Meta-scheduler

The GridWay Meta-scheduler. Contents. What is GridWay? Architecture Components Scheduling Policies Examples of Grid Deployments Enterprise Grids Partner Grids A Tool for Interoperability. For the user A LRM-like environment for submitting, monitoring, and controlling jobs

yachi
Download Presentation

The GridWay Meta-scheduler

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The GridWay Meta-scheduler

  2. Contents • What is GridWay? • Architecture • Components • Scheduling Policies • Examples of Grid Deployments • Enterprise Grids • Partner Grids • A Tool for Interoperability

  3. For the user A LRM-like environment for submitting, monitoring, and controlling jobs For the developer An standard-base development framework for Grid Applications For the sysadmin A policy-driven job scheduler User-side Grid Accounting For the Grid architect / solution provider A modular component to use different infrastructures A key component to deploy different Grids (enterprise, partner, utility…) What is GridWay? The GridWay meta-scheduler is a scheduler virtualization layer on top of basic Globus services (GRAM, MDS & GridFTP)

  4. GridWay Architecture DRMAA • LRM-like Command Line Interface • OGF DRMAA C & JAVA Bindings • JSDL (Posix & HTC profiles) • Array jobs, DAG workflows and MPI jobs $> Applications CLI Application-Infrastructure decoupling .C, .java .C, .java • Advanced (Grid-aware) scheduling policies • Fault detection & recovery Grid Meta- Scheduler • Straightforwarddeployment(basic services) • Globus-based infrastructures • Component to deploy different Grids Globus Services Grid Middleware • highly dynamic & heterogeneous • high fault rate Infrastructure PBS SGE

  5. MDS2 MDS2 GLUE MDS4 GridFTP RFT pre-WS GRAM WS GRAM Components DRMAA library CLI Job Submission Job Monitoring Job Control Job Migration GridWay Core Request Manager Job Pool Host Pool Dispatch Manager Scheduler Transfer Manager Execution Manager Information Manager Job Preparation Job Termination Job Migration Resource Discovery Resource Monitoring Information Services File Transfer Services Execution Services

  6. Scheduling Policies • Rank Expressions • Fixed Priority • User Usage History • Failure Rate Resource Policies Grid Scheduling = Job + Resource Policies Matching Resources for each job (user) Pending Jobs Job Policies • Fixed Priority • Urgent Jobs • User Share • Deadline • Waiting Time

  7. Enterprise Grids Characteristics • “Small” scale infrastructures (campus/enterprise) with one meta-scheduler instance • Resources within the same administration domain that may be running different LRMS and be geographically distributed • Goal & Benefits • Integrate heterogeneous systems • Improve return of IT investment • Performance/Usage maximization

  8. Applications • DRMAA interface • Portal • Command Line Interface Users • One meta-scheduler • Grid-wide policies GridWay Globus Globus Globus Middleware SGE Cluster PBS Cluster LSF Cluster Infrastructure Enterprise Grids Architecture Examples • European Space Astronomy Center • Data Analysis from space missions • DRMAA • UABGrid, University of Alabama • Bioinformatics applications

  9. Partner Grids Characteristics • “Large” scale infrastructures with one or several meta-schedulers • Resources belong to different administrative domains Goal & Benefits • Large-scale, secure and reliable sharing of resources • Support collaborative projects • Access to higher computing power to satisfy peak demands

  10. Partner Grids Architecture Examples (Virtual) Organization • EGEE-II • gLite-LHC interoperability • Virtual Organizations • Fusion: Massive Ray Tracing • Biomed: CD-HIT (Worflow) Applications • DRMAA interface • Science Gateways Users Users • Multiple metaschedulers • (V)Organization-wide policies GridWay GridWay • AstroGrid-D, German Astronomy Community Grid • Supercomputing resources • Astronomy-specific resources • GRAM interface Middleware Globus Globus Globus SGE Cluster PBS Cluster LSF Cluster • Multiple Admin. Domains • Multiple Organizations Infrastructure

  11. A Tool for Interoperability • Different Middlewares (e.g. WS and pre-WS) • Different Data/Execution architectures • Different Information models • Integration through adapters • Global DN’s • Demo in June 2007 at TeraGrid07 Users GridWay gLite Globus/WS Globus/WS gLite Globus/WS Globus/WS SGE Cluster PBS Cluster SGE Cluster PBS Cluster SGE Cluster PBS Cluster

More Related