1 / 34

Use of Performance Prediction Techniques for Grid Management

Use of Performance Prediction Techniques for Grid Management. Junwei Cao University of Warwick April 2002. Outline. Introduction Performance Prediction Local Grid Scheduling Global Grid Management A Case Study Conclusions & Future Work. Introduction. Glossary Overview Related Works.

jessamine-bentley
Download Presentation

Use of Performance Prediction Techniques for Grid Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Use of Performance Prediction Techniques for Grid Management Junwei Cao University of Warwick April 2002

  2. Outline • Introduction • Performance Prediction • Local Grid Scheduling • Global Grid Management • A Case Study • Conclusions & Future Work

  3. Introduction Glossary Overview Related Works

  4. Glossary • Grids – computational grids • Resources – multiprocessors or clusters • Applications (tasks) – MPI & PVM parallel programs • Users – developers and end users • Agents, Requests & Services • Performance – execution time

  5. Application Tools Global Grid Management Performance Evaluation Engine Local Grid Scheduling Resource Tools Overview Grid Users Grid Resources

  6. Related Works • Performance evaluation • POEMS, AppLeS, … • Local grid schedulers • Condor, LSF, Ninf, Nimrod, … • Global grid management • Globus, Legion, DPSS, …

  7. Performance Prediction Methodology Implementation

  8. Model Parameters Application Layer Subtask Layer Parallel Template Layer Hardware Layer Predicted Execution Time PACE Methodology acts as the entry point to the performance study describes the sequential parts within an application describes the parallel characteristics of subtasks characterises the comm. and comp. abilities of a particular system

  9. Source Code Analysis Object Editor Object Library HMCL Compiler PSL Compiler CPU Network (MPI, PVM) Cache (L1, L2) PACE Toolkit Application Tools Evaluation Engine Resource Tools

  10. Summary • Advantages • Reasonable accuracy • Rapid evaluation time • Easy cross-platform comparisons • Limitations • Application source codes required • Static resource configurations

  11. Local Grid Scheduling Algorithms Implementation (Titan)

  12. 2n-1 FIFO Algorithm Processor 1 Processor 2 Processor 3 Processor 4 Processor 5 Processor 6 Processor 7 Processor 8

  13. Genetic Algorithm • Heuristic • Evolutionary • Near-optimal: • Makespan • Idletime • Deadlines

  14. Titan Implementation Requests Results Service Communication Module Task Management Task Execution Resource Monitoring GA Scheduling PACE Evaluation Engine

  15. Global Grid Management Methodology Implementation (ARMS) Metrics

  16. A A A A Agent-based Methodology • Agent structure • Communication layer • Decision-making layer • Local management layer • Agent hierarchy • Service advertisement • Service discovery • Agent Capability Tables A User

  17. M Optimisation Strategies • Advertisement • Data-push & data-pull • Periodic & event-driven • Discovery • Local services • Services in ACTs • Lower or upper agents • Optimisation • Modelling • Simulation A User A A A A

  18. M A A A A A T T T T T ARMS Implementation • Service information • PACE models • Makespan • Request information • Application binary • PACE model • Deadline • Matchmaking • Estimation (FIFO) • Deadline User

  19. Metrics • Average advance time of application execution completions (compared to required deadlines) • Average processor utilisation rate – utilisedtime/totaltime • Load balancing level – mean square deviation of processor utilisation rates • Average discovery speed – #req./#disc.conn. • Average discovery efficiency – #req./(#disc.conn.+ #adver.conn.)

  20. A Case Study Design Demonstrations Results

  21. S1 (SGIOrigin2000, 16) S2 (SGIOrigin2000, 16) S5 (SunUltra5, 16) S4 (SunUltra10, 16) S3 (SunUltra10, 16) S12 (SunSPARCstation2, 16) S10 (SunUltra1, 16) S8 (SunUltra1, 16) S6 (SunUltra5, 16) S11 (SunSPARCstation2, 16) S7 (SunUltra5, 16) S9 (SunUltra1, 16) Experiment Design sweep3d fft improc closure jacobi memsort cpi

  22. FIFO FIFO FIFO FIFO FIFO Experiment 1

  23. FIFO FIFO FIFO FIFO FIFO Experiment 1

  24. GA GA GA GA GA Experiment 2

  25. GA GA GA GA GA Experiment 3

  26. Application Execution Both GA and agents contribute towards the improvement in application executions.

  27. Resource Utilisation S11 & S12 benefit mainly from the GA. S1 & S2 benefit mainly from agents.

  28. Load Balancing The GA contributes more to local grid load balancing. Agents contribute more to global grid load balancing.

  29. Discovery Speed & Efficiency Discovery speed (*100) No advertisement: Low speed Low efficiency Reasonable advertisement: High speed High efficiency Discovery efficiency (*100) Too much advertisement: Very high speed Very low efficiency

  30. Conclusions & Future Work

  31. Conclusions • Performance prediction capabilities are essential to grid management. • Genetic algorithm is applied for local grid scheduling. • Global grid management is achieved using an agent-based methodology. • Agent-based framework is scalable, flexible, extensible and easy for further enhancement.

  32. Future Work • Impact of prediction accuracy on grid management and scheduling • Transaction-based application performance modelling • Integration with Globus and NWS • More than discovery, enabling negotiation and coordination

  33. Agent Coordination Performance Prediction PACE Agent Negotiation Application Scheduling AppLeS Agent Discovery Information Service Globus Grid Monitoring NWS Agent Communication Knowledge Representation … … JAG: Java Agents for Grids

  34. Questions are welcome …

More Related