1 / 37

Superscheduling and Resource Brokering

Superscheduling and Resource Brokering. Sven Groot (0024821). Grid Information Service. Not all information available Grid Information System Globus Monitoring and Discovery Service (MDS2) Grid Monitoring Architecture (GMA) Common features Organise sensors Static vs. Dynamic data

angeni
Download Presentation

Superscheduling and Resource Brokering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Superscheduling and Resource Brokering Sven Groot (0024821)

  2. Grid Information Service • Not all information available • Grid Information System • Globus Monitoring and Discovery Service (MDS2) • Grid Monitoring Architecture (GMA) • Common features • Organise sensors • Static vs. Dynamic data • Extensible • Agreed upon schema

  3. Stages of Grid Scheduling • Phase 1: Resource Discovery • Authorization filtering • Application Requirement Definition • Minimal requirement filtering

  4. Stages of Grid Scheduling (2) • Phase 2: System Selection • Dynamic information gathering • System Selection

  5. Stages of Grid Scheduling (3) • Phase 3: Job Execution • Advance Reservation (optional) • Job Submission • Preparation Tasks • Monitoring Progress • Job Completion • Cleanup Tasks

  6. Application requirements

  7. Application Requirements (2) • General requirements • Compute-related requirements • Data-related requirements • Network-related requirements

  8. Application Requirements (3) • Challenges • Application deployment • Metacomputing • Predicting performance • Theoretical prediction • History based prediction • Testcase-based prediction • Adaptive brokering

  9. Application Requirements (4) • Related issues • Application frameworks • Virtual Organizations • Security requirements • Accounting policies • User preferences

  10. Scheduling in GrADS • Scheduling phases • Launch-time scheduling • Rescheduling • Meta-scheduling

  11. GrADS

  12. GrADS (2) • Focus applications • ScaLAPACK • Cactus • FASTA • Iterative applications • Jacobi method • Game of Life • Fish

  13. GrADS: Launch-time scheduling

  14. GrADS: Launch-time scheduling (2) • Configurable Object Program • Application requirements definition • AART • ClassAds • Redline

  15. ClassAds sample

  16. GrADS: Launch-time scheduling (3) • Performance model • General method • develop an analytic model for well-understood aspects of applicatio or system performance • test the analytic model against achieved application performance • develop empirical models for poorly-understood aspects of application or system behavior • Some application specific methods • Implemented as shared libraries

  17. GrADS: Launch-time scheduling (4) • Mapper • Maps data and/or tasks to resources • Different mapping methods • Equal allocation • Time balancing • Data locality

  18. GrADS: Launch-time scheduling (5) • Search procedure • General steps • identify a large number of sets of resources that may be good platforms for the application • use the application-specific mapper and performance model to generate a data map and predicted execution time for those resource sets • select the resource set that results in the lowest predicted execution time

  19. GrADS: Launch-time scheduling (6) • Resource-aware search

  20. GrADS: Launch-time scheduling (6) • Simulated Annealing

  21. GrADS: Rescheduling • Additional complexities • Lack of built-in mechanisms • Need to distinguish processors that are running/not running the current process • Overheads can be high

  22. GrADS: Rescheduling (2)

  23. GrADS: Rescheduling (2) • Rescheduling methods • Application migration • Process swapping

  24. GrADS: Metascheduling

  25. Grid Service Level Agreements • Contract • Provide some capability • Perform some task • Types of SLAs • Resource Service Level Agreements • Task Service Level Agreements • Binding Service Level Agreements

  26. Grid SLAs (2)

  27. Grid SLAs (3) • Motivating scenarios • Community Scheduler Scenario

  28. Grid SLAs (4) • Motivating scenarios (cont’d) • File transfer scenario

  29. Grid SLAs • Resource virtualization

  30. Multicriteria • Basic definitions • Pareto Dominance • Pareto Optimality • Pareto-optimal set • Pareto Front

  31. Multicriteria (2) • Motivations • Various stakeholders and their preferences • Job scheduling • Application-Level scheduling • Hard constraints and soft constraints

  32. Multicriteria (3) • Approach • Criteria • Related to stakeholders • Related to entire system • Time criteria • Cost criteria • Resource utilization criteria • Modeling preferences

  33. Multicriteria (4) • Selection method • Rule-based system requirements • Expression of policies • Execution of different scheduling procedures • Adaptation to the environment • Selection of the best solution • Multicriteria optimization

  34. Example (cont’d)

  35. Example (cont’d) • Aggregate criteria • End user satisfaction • Resource Owner Satisfaction • VO overall performance

  36. Example (cont’d)

More Related