180 likes | 313 Views
Flexible QoS-Based Service Selection in Large-Scale Grids. Sebastian Stein, Terry R. Payne, Nicholas R. Jennings {ss2,trp,nrj}@ecs.soton.ac.uk. UK e-Science All Hands Meeting HPC Grids of Continental Scope, 9 September, 2008. Outline. Background & Motivation
E N D
Flexible QoS-Based Service Selection in Large-Scale Grids Sebastian Stein, Terry R. Payne, Nicholas R. Jennings {ss2,trp,nrj}@ecs.soton.ac.uk UK e-Science All Hands Meeting HPC Grids of Continental Scope, 9 September, 2008
Outline • Background & Motivation • Decision-Theoretic Service Selection • Empirical Evaluation • Conclusions
Background • Large-scale Grids allow scientists access to a vast range of distributed services. • Often many services are composed as part of larger workflows. Pegasus Workflow myGrid Workflow Sources: Exploring Williams-Beuren Syndrome Using myGrid, Hannah Tipney, http://www.mygrid.org.uk/wiki/pub/Mygrid/PresentationStore/ISMB04_Glasgow.ppt,, Pegasus Teragrid Talk SC2005 Seattle Washington, http://pegasus.isi.edu/pegasus/presentations/pegasus-tg-final.ppt
Service Selection • Services are dynamically selected at run-time. -£25 -£20 -£10 £100 24 h Failure! -£5 • Services are provided by autonomous agents. • These may be unreliable (may fail and or take longer than expected)… • …and heterogeneous.
Problem Statement • How to design a service selection algorithm... • ... that deals with unreliable and heterogeneous service providers, • ... that is effective and efficient, • ... that makes autonomous decisions on the user‘s behalf.
Central Idea • How to address uncertainty during service selection? Existing work relies on single service for each workflow task. We can do better by using redundancy and contingencyplans. … and by taking into consideration service heterogeneity.
Service Model • We devised an abstract model to describe a service-oriented Grid. • Utility function:
Flexible Selection • We want to find a service allocation for each task, e.g.: • Decision-theoretic approach to maximise expected utility: Expected cost Expected reward
Flexible Selection • This is an NP-hard problem and even calculating the expected utility is intractable. • Hence: Approximate the expected utility of an allocation using a heuristic utility function: • This is optimised using a local search.
Local Task Calculations • We start by calculating a number of performance parameters for each task in the workflow: 2 Service populations: Success Probability: 95.00% Expected Cost: £30.00 Expected Duration: 60.50 min Variance: 361.19 min2 Success Probability: 96.83% Expected Cost: £7.23 Expected Duration: 75.59 min Variance: 4030.84 min2 Success Probability: 99.99% Expected Cost: £26.15 Expected Duration: 56.23 min Variance: 1501.57 min2
Global Workflow Calculations • These task parameters are then combined to estimate the overall expected profit: 7 30 95% 99% £10 £5 Global Parameters: Success Probability: 68% Estimated Cost: £98.40 Estimated Duration: 132 min Variance: 912 min2 100% 52 £24 43 90% £30 266 25 22 80% 15 £3 100% £42 111 345 164 292 = 162.48 383.64 0.68 98.40
Empirical Evaluation • To test our algorithm, we compare it to existing approaches in a simulated Grid system: • Naïve: Selects a single provider for each task. • Global QoS: Optimises weighted QoS measures over entire workflow with deadline and budget constraints. • Adaptive Global QoS: As above, but also uses timeouts and re-selects services dynamically.
Empirical Evaluation Consistently outperforms other approaches Positive utility even when uncertainty high Avoids overall loss
Conclusions • As Grids become larger, service uncertainty and heterogeneity must be considered. • This uncertainty can have a significant and detrimental impact, even when the individual failure probability of services is low. • We developed a flexible service selection algorithm that uses redundancy and contingency plans to achieve a high average profit, even when individual services are highly unreliable.
Any Questions? Thank you! • More information: • Stein, S. (2008). Flexible Service Provisioning in Multi-Agent Systems, PhD Thesis, University of Southampton. • - Stein, S., Jennings, N.R., and Payne, T.R. (2008). Flexible Service Provisioning with Advance Agreements. In: Proc. AAMAS-08, pp. 249-256. This work was sponsored by: