1 / 27

Connection Scheduling in Web Servers

Connection Scheduling in Web Servers. Mor Harchol-Balter School of Computer Science Carnegie Mellon harchol@cs.cmu.edu. FCFS. jobs. jobs. PS. SRPT. jobs. Q: Which minimizes mean response time?. “size” = service requirement. load r < 1. Q: Which best represents

nusa
Download Presentation

Connection Scheduling in Web Servers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Connection Scheduling in Web Servers Mor Harchol-Balter School of Computer Science Carnegie Mellon harchol@cs.cmu.edu

  2. FCFS jobs jobs PS SRPT jobs Q: Which minimizes mean response time? “size” = service requirement load r < 1

  3. Q: Which best represents scheduling in web servers ? FCFS jobs “size” = service requirement load r < 1 jobs PS SRPT jobs

  4. How about using SRPT in Web servers as opposed to the traditional processor-sharing (PS) type scheduling ?

  5. Immediate Objections 1) Can’t assume known job size Many servers receive mostly static web requests.“GET FILE”For static web requests, know file size Approx. know service requirement of request. 2) But the big jobs will starve ...

  6. THEORY IMPLEMENT Outline of Talk 1) “Analysis of SRPT Scheduling: Investigating Unfairness” Bansal and Harchol-Balter 2) “Implementation of SRPT Scheduling in Web Servers” Harchol-Balter and Bansal and Schroeder

  7. THEORY SRPT has a long history ... 1966 Schrage & Miller derive M/G/1/SRPT response time: 1968 Schrage proves optimality 1979 Pechinkin & Solovyev & Yashkov generalize 1990 Schassberger derives distribution on queue length BUT WHAT DOES IT ALL MEAN?

  8. THEORY SRPT has a long history (cont.) 1990 - 97 7-year long study at Univ. of Aachen under Schreiber Evaluate mean response time under SRPT for specific distributions ……….. SRPT WINS BIG ON MEAN! 1997, 1998 Variations on SRPT: Evaluation via simulation only: Mudiano, Cherkasova, etc. 1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen 1998 Bender & Chakrabarti & Muthukrishnan: Theoretical results under adversarial arrival sequence: SRPT STARVES BIG JOBS! Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs ... Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”

  9. I SRPT THEORY Counter-intuitive! Our Analytical Results (M/G/1): All-Can-Win Theorem: Under heavy-tailed workloads, ALL jobs, including the very biggest, prefer SRPT to PS, provided load not too close to 1. Almost-All-Win-Big Theorem: Under heavy-tailed workloads, 99% of all jobs perform significantly better under SRPT. PS SRPT

  10. THEORY Our Analytical Results (M/G/1): Moderate-Load Theorem: If load < .5, for every job size distribution, ALL jobs prefer SRPT to PS. Bounding-the-damage Theorem: For any load, for every job size distribution, for every size x, æ ö r ç ÷ < + E [ T ( x )] 1 E [ T ( x )] ç ÷ SRPT PS 2(1 - r ) è ø For more theorems, see: www.cs.cmu.edu/~harchol

  11. IMPLEMENT From theory to practice: What does SRPT mean within aWeb server? • Many devices:Where to do the scheduling? • No longer one job at a time.

  12. IMPLEMENT Previous work on implementation: 1998 Almeida, Dabu, Manikutty, Cao: Prioritizing HTTP requests at Web servers. “Nice” the low-priority process. 1999 Crovella, Frangioso, Harchol-Balter: SRPT scheduling of requests. User-level scheduling of reads and writes. What is the problem with both of these?

  13. Web Server Network/O.S. insides of traditional Web server IMPLEMENT Socket 1 Client1 Network Card Socket 2 Client2 BOTTLENECK Client3 Socket 3 Sockets take turns draining --- FAIR = PS.

  14. IMPLEMENT Web Server Network/O.S. insides of our improved Web server Socket 1 Client1 S Network Card 1st Socket 2 Client2 2nd M BOTTLENECK 3rd Client3 Socket 3 L priority queues. Socket corresponding to file with smallest remaining data gets to drain first.

  15. IMPLEMENT Experimental Setup 1 1 FLASH WEB SERVER CLIENTS 2 2 3 3 Direct Connection 10Mb/s 200 200 Linux 0.S. Linux 0.S. • Implement SRPT-based scheduling: • 1) Modifications to Linux O.S.: 5 priority Levels • 2) Modifications to Flash Web server • 3) Priority algorithm design.

  16. IMPLEMENT Experimental Setup 1 1 FLASH WEB SERVER 2 2 CLIENTS 3 3 200 200 Linux 0.S. Linux 0.S. • Comparison of SRPT implementation vs. FAIR scheduling standard. • Comparisons done under trace-based workload and under • Web workload generator. • Under trace-based workload: • -- Number requests made: 90,000 • -- Size of file requested: 41B -- 2 MB • -- File sizes requested have heavy-tailed distribution. • Experiment under range of loads.

  17. IMPLEMENT Preliminary Comments to Results: 1 1 FLASH WEB SERVER 2 2 CLIENTS 3 3 200 200 Linux 0.S. Linux 0.S. • Measured job throughput, byte throughput, and bandwidth • utilization were same under SRPT and FAIR scheduling. • Same set of requests complete. • No additional CPU overhead under SRPT scheduling. • CPU utilization always 1% - 5%. • Network was bottleneck in all experiments.

  18. Results: Mean Response Time Mean Response Time (ms) FAIR SRPT Load

  19. Results: Mean Slowdown Mean Slowdown FAIR SRPT Load

  20. Results: Mean Response Time vs. Size Load = 0.8 Mean Response Time (ms) FAIR SRPT Requested File Size (bytes)

  21. Mean Response Time vs. Size Percentile Load =0.8 FAIR Mean Response time (ms) SRPT Percentile of Request Size

  22. Variance in Response Time vs. Size Percentile Load =0.8 FAIR Variance in Response Time Variance in Response Time SRPT Percentile of Request Size Percentile of Request Size

  23. Details, Optimizations, and Shortcuts • Diffserv implementation headaches. • The problem of high startup overhead: • Scheduling Solutions! • All the benefit with much less work: • QUICKFIX.

  24. Results: Mean Response Time vs. Size Load = 0.8 FAIR Mean Response Time (ms) QUICKFIX Percentile of Request Size (bytes)

  25. Results: Mean Response Time vs. Size Load = 0.8 FAIR Mean Response Time (ms) QUICKFIX Requested File Size (bytes)

  26. Conclusions • SRPT scheduling yields significant improvements in Mean Response Time at the server. • Negligible starvation. • No CPU overhead. • No drop in throughput.

  27. Big Future Work • Handling propagation delay and slow clients. • Non-static requests. • Smart routers. • Overload.

More Related