1 / 54

Web Server Request Scheduling

Web Server Request Scheduling. Mingwei Gong Department of Computer Science University of Calgary November 16, 2004. Outline. Introduction and Background Quantifying the Properties of SRPT Policy Evaluating the Sensitivity to the Arrival Process and Job Size Distribution

theo
Download Presentation

Web Server Request Scheduling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Web Server Request Scheduling Mingwei Gong Department of Computer Science University of Calgary November 16, 2004

  2. Outline • Introduction and Background • Quantifying the Properties of SRPT Policy • Evaluating the Sensitivity to the Arrival Process and Job Size Distribution • Evaluating Hybrid SRPT

  3. Introduction • User-perceived Web response time is composed of several components: • Transmission delay, propagation delay in network • Delays caused by TCP protocol effects (e.g., handshaking, slow start, packet loss, retxmits) • Queueing delays at the Web server itself, which may be servicing 100’s or 1000’s of concurrent requests • Queueing delays at busy routers • Our focus in this work: Web request scheduling

  4. Scheduling Policies • FCFS: First Come First Serve • PS: Processor Sharing • SRPT: Shortest Remaining Processing Time • LAS: Least Attained Service • FSP: Fair Sojourn Protocol

  5. jobs FSFS FCFS • FCFS: First Come First Serve • typical policy for single shared resource (“unfair”) • e.g., drive-thru restaurant • e.g., routers

  6. jobs 1/M PS PS • PS: Processor Sharing • time-sharing a resource amongst M jobs • each job gets 1/M of the resources (equal, “fair”) • e.g., CPU; VM; multi-tasking; Apache Web server

  7. jobs SRPT SRPT jobs SRPT • SRPT: Shortest Remaining Processing Time • pre-emptive version of Shortest Job First (SJF) • give resources to job that will complete quickest • e.g., ??? (express lanes in grocery store)(almost)

  8. SRPT (cont’d) • SRPT is well known to be optimal in terms of mean response time among all work-conserving scheduling policies. • But, it is rarely deployed in the applications, mainly due to two concerns: • Unfairness problem, that is, large jobs may be penalized. • Unknown job sizes beforehand

  9. FSP • FSP: Fair Sojourn Protocol • order the jobs according to the PS policy • give full resources to job that with the earliest PS completion time Eric J. Friedman, Shane G. Henderson: “Fairness and efficiency in web server protocols”. SIGMETRICS 2003: 229-237

  10. LAS • LAS: Least Attained Service scheduling • At any time, LAS gives service to a job that has received the least amount of service. • A size based policy that favors short jobs without knowing their sizesin advance Rai, I. A., Urvoy-Keller G., and Biersack, E. W., “Analysis of LAS scheduling for job size distributions with high variance. “ Proceedings of ACM Sigmetrics 2003, San Diego, June 2003. Rai, I. A., Urvoy-Keller, G., Vernon, M., and Biersack, E. W., “Performance models for LAS-based scheduling disciplines in a packet switched network”, Proceedings of ACM Sigmetrics 2004

  11. Quantifying the Properties of SRPT Policy

  12. Related Work • Theoretical work: • SRPT is provably optimal in terms of mean response time (“classical” results) • Practical work: • CMU: prototype implementation in Apache Web server. The results are consistent with theoretical work.

  13. Related Work (Cont’d) • Harchol-Balter et al. show theoretical results: • For the largest jobs, the slowdown asymptotically converges to the same value for any preemptive work-conserving scheduling policies (i.e., for these jobs, SRPT, or even LRPT, is no worse than PS) • For sufficiently large jobs, the slowdown under SRPT is only marginally worse than under PS, by at most a factor of 1 + ε, for small ε > 0. [M.Harchol-Balter, K.Sigman, and A.Wierman 2002], “Asymptotic Convergence of Scheduling Policies w.r.t. Slowdown”, Proceedings of IFIP Performance 2002, Rome, Italy, September 2002

  14. “asymptotic convergence” “crossover region” (mystery hump) 1 1-p x y A Pictorial View 8 PS Slowdown SRPT 1 0 8 Job Size

  15. Research Questions • Do these properties hold in practice for empirical Web server workloads? (e.g., general arrival processes, service time distributions) • What does “sufficiently large” mean? • Is the crossover effect observable? • If so, for what range of job sizes? • Is PS (the “gold standard”) really “fair”?

  16. Overview of Research Methodology • Trace-driven simulation of simple Web server • Empirical Web server workload trace (WorldCup’98) for main experiments • Probe-based sampling methodology • Estimate job response time distributions for different job size, load level, scheduling policy

  17. Performance Metrics • Number of jobs in the system • Slowdown: • The slowdown of a job is its observed response time divided by the ideal response time if it were the only job in the system • Ranges between 1 and  • Lower is better

  18. Empirical Web Server Workload

  19. PS PS PS Probe-Based Sampling Algorithm Slowdown (1 sample) Repeat N times

  20. Load 50% Load 80% Load 95% Example Results for 3 KB Probe Job

  21. Load 50% Load 80% Load 95% Size 100K Example Results for 100 KB Probe Job

  22. Load 50% Load 80% Load 95% Example Results for 10 MB Probe Job

  23. Statistical Summary of Results

  24. Two Aspects of Unfairness • Endogenous unfairness: (SRPT) • Caused by an intrinsic property of a job, such as its size. This aspect of unfairness is invariant • Exogenous unfairness: (PS) • Caused by external conditions, such as the number of other jobs in the system, their sizes, and their arrival times.

  25. PS is “fair” Sort of! Exogenous unfairness dominant Observations for PS

  26. Endogenous unfairness dominant Observations for SRPT

  27. Asymptotic Convergence? Yes!

  28. Linear Scale Log Scale Illustrating the crossover effect (load=95%) 3M 3.5M 4M

  29. Crossover Effect? Yes!

  30. Summary and Conclusions • Trace-driven simulation of Web server scheduling strategies, using a probe-based sampling methodology (probe jobs) to estimate response time (slowdown) distributions • Confirms asymptotic convergence of the slowdown metric for the largest jobs • Confirms the existence of the “cross-over effect” for some job sizes under SRPT • Provides new insights into SRPT and PS • Two types of unfairness: endogenous vs. exogenous • PS is not really a “gold standard” for fairness!

  31. Evaluating the Sensitivity to Arrival Process and Job Size Distribution

  32. Research Questions • What is the impact from different arrival process and job size distribution? • How the crossover region will be affected? Does it depend on the arrival process and the service time distribution? If so, how?

  33. Effect of Request Arrival Process • Using fixed size transfers • 3 KB in this experiment • Changing the Hurst parameter • from 0.50 to 0.90

  34. Marginal Distr. Of Number of Jobs in the System (p=0.8) Hurst 0.5 Hurst 0.7 Hurst 0.9

  35. Marginal Distr. Of Number of Jobs in the System (p=0.95) Hurst 0.7 Hurst 0.9 Hurst 0.5

  36. Mean Performance under Load 0.80

  37. Mean Performance under Load 0.95

  38. Sensitivity to Arrival Process • A bursty arrival process (e.g., self-similar traffic, with Hurst parameter H > 0.5) makes things worse for both PS and SRPT policies • A bursty arrival process has greater impact on the performance of PS than on SRPT

  39. Effect of Heavy-tailed Job Size Distribution • Using Deterministic Arrival Process • Adjusting the heavy-tailed parameter • From 1.0 to 2.0

  40. Sensitivity to Job Size Distribution • SRPT loves heavy-tailed distributions: the heavier the tail the better! • For all Pareto parameter values and all system loads considered, SRPT provides better performance than PS with respect to mean slowdown and standard deviation of slowdown

  41. The Crossover Effect Revisited • The crossover region tends to get smaller as the burstiness of the arrival process increases • PS performs much worse under bursty arrival process • SRPT can still manage a relatively good performance

  42. Evaluating Hybrid SRPT

  43. Research Questions • Efficiency and Fairness, how to achieve both? • Can we do better? If so, how?

  44. PS, FSP and SRPT

  45. K-SRPT • Multi-threaded version of SRPT, that allows up to K jobs (the K smallest RPT ones) to be in service concurrently (like PS), though with the same fixed aggregate service rate. Additional jobs (if any) in the system wait in the queue. And of course it is preemptive, like SRPT. • share = Min (J, K) • If J > K, then first K jobs each receives 1/share • Else, those J jobs, each receives 1/share

  46. Simulation Results for K-SRPT Jobs in System for K-SRPT Slowdown Profile Plot for K-SRPT

  47. T-SRPT • Determining whether the system is "busy" or not depends on a threshold T for the number of jobs (J) in the system. share = Min (J, K) • If J > T, then use SRPT • Else, use PS.

  48. Simulation Results for T-SRPT Jobs in System for T-SRPT Slowdown Profile Plot for T-SRPT

  49. DT-SRPT • Double-Threshold SRPT uses two threshold: • A high threshold T_high at which the policy switches from PS to SRPT. • A low threshold T_low at which it switches back from SRPT to PS.

  50. Simulation Results for DT-SRPT Jobs in System for DT-SRPT Slowdown Profile Plot for DT-SRPT

More Related