1 / 12

Scheduler Basics

Scheduler Basics. Juli Rew CISL User Forum May 19, 2005. IBM Scheduling Life of a Job Submit Filter Batch Priority Scheduler Factors Affecting BPS Job Scheduling LoadLeveler Load Sharing Facility Scheduling • LSF Scheduling on Linux Systems

goldy
Download Presentation

Scheduler Basics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scheduler Basics Juli Rew CISL User Forum May 19, 2005

  2. IBM Scheduling Life of a Job Submit Filter Batch Priority Scheduler Factors Affecting BPS Job Scheduling LoadLeveler Load Sharing Facility Scheduling • LSF Scheduling on Linux Systems Differences from IBM Scheduling Overview

  3. IBM Scheduling: Life of a Job llsubmit job Submit Filter Requirements Processing BPS Job Ordering LoadLeveler Job Execution Requirements Not Met Reject Job Build Ordered List of Jobs Job Starts Requirements Problem Staff Rejects Job Job Completes Done Done Done

  4. Checks the LoadLeveler job script for: - valid parameters - valid queue name - consistent combinations of features, eg., shared/not_shared, tasks_per_node/node options Moves jobs with allocation holds to hold queues Moves jobs with cutoff projects to standby queue Submit Filter Features

  5. Written at NCAR Orders jobs based on policy Creates separate facilities (Community, Climate System Laboratory) Further separates jobs into proposal groups (NCAR/UNIV, CCSM/oCSL) Hands the final order list to LoadLeveler Allows for backfilling of jobs to avoid idle resources Batch Priority Job Scheduler Features

  6. Bluesky Queue Priorities

  7. all_spec jobs run with the highest priority and can access all nodes Below that, all com and csl jobs divided equally Round Robin by Group/User ------------------ all_spec ------------------ com csl \ / top job 50-50% split not hard Prioritization of Jobs by BPS

  8. Backfilling - Jobs that will not interfere with start of highest priority job allowed to slip in - Sweet spot: < 3 hours and small node count Allocation Holds - Job flagged if a project/division exceeds its 30-day or 90-day allocation thresholds - H1 and H2 jobs reordered at a priority above standby but below non-flagged jobs Special Initiatives - Nodes reserved for real-time or other special runs Other Factors Affecting Job Scheduling

  9. batchview command gives snapshot of current ordering Basic information on scheduling given at http://www.scd.ucar.edu/docs/ibm/ref/llsched.html Documentation and Utilities

  10. IBM's batch control job system Allows jobs to be started, stopped, or cancelled Controls allocation of resources (CPU, memory) Allows custom scheduler plug-in (e.g., BPS) Two mutually-exclusive options: LoadLeveler scheduler or custom scheduler. LoadLeveler

  11. Commercial product from Platform Computing Currently being used on major Linux platforms Also available for IBM, but still in evaluation Ability to do Hierarchical Fair-Share Scheduling with Backfill, based on same facility scheme used in BPS Community/CSL facility division implemented implicitly within the scheduler rather than explicitly by queue name • Can schedule among multiple platforms - "Grid” Load Sharing Facility

  12. Questions?

More Related