1 / 35

Mor Harchol-Balter Carnegie Mellon

Scheduling Your Network Connections. Mor Harchol-Balter Carnegie Mellon. with Mukesh Agrawal. with Bianca Schroeder. with Nikhil Bansal. FCFS. jobs. jobs. PS. SRPT. jobs. Q: Which minimizes mean response time?. “size” = service requirement. load r < 1.

julio
Download Presentation

Mor Harchol-Balter Carnegie Mellon

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scheduling Your Network Connections Mor Harchol-Balter Carnegie Mellon with Mukesh Agrawal with Bianca Schroeder with Nikhil Bansal

  2. FCFS jobs jobs PS SRPT jobs Q: Which minimizes mean response time? “size” = service requirement load r < 1

  3. Q: Which best represents scheduling in web servers ? FCFS jobs “size” = service requirement load r < 1 jobs PS SRPT jobs

  4. How about using SRPT in Web servers as opposed to the traditional processor-sharing (PS) type scheduling ?

  5. Immediate Objections 1) Can’t assume known job size Many servers receive mostly static web requests.“GET FILE”For static web requests, know file size Approx. know service requirement of request. 2) But the big jobs will starve ...

  6. THEORY IMPLEMENT Outline of Talk 1) “Analysis of SRPT Scheduling: Investigating Unfairness” with Nikhil Bansal 2) “Implementation of SRPT Scheduling in Web Servers” with Nikhil Bansal and Bianca Schroeder and Mukesh Agrawal

  7. THEORY SRPT has a long history ... 1966 Schrage & Miller derive M/G/1/SRPT response time: 1968 Schrage proves optimality 1979 Pechinkin & Solovyev & Yashkov generalize 1990 Schassberger derives distribution on queue length BUT WHAT DOES IT ALL MEAN?

  8. THEORY SRPT has a long history (cont.) 1990 - 97 7-year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN! 1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS! Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs ... Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”

  9. I SRPT THEORY Counter-intuitive! Our Analytical Results (M/G/1): All-Can-Win Theorem: Under workloads with heavy-tailed (HT) property, ALL jobs, including the very biggest, prefer SRPT to PS, provided load not too close to 1. Almost-All-Win-Big Theorem: Under workloads with HT property, 99% of all jobs perform orders of magnitude better under SRPT. PS SRPT Many more such results in Sigmetrics talk.

  10. THEORY Our Analytical Results (M/G/1): Moderate-Load Theorem: If load < .5, for every job size distribution, ALL jobs prefer SRPT to PS. Bounding-the-damage Theorem: For any load, for every job size distribution, for every size x, æ ö r ç ÷ < + E [ T ( x )] 1 E [ T ( x )] ç ÷ SRPT PS 2(1 - r ) è ø

  11. What’s the Heavy-Tail property? Defn: heavy-tailed distribution: - a > < a < Pr { X x } ~ x , 0 2 Many real-world workloads well-modeled by truncated HT distribution. Key property: HT Property: “Largest 1% of jobs comprise half the load.”

  12. IMPLEMENT From theory to practice: What does SRPT mean within aWeb server? • Many devices:Where to do the scheduling? • No longer one job at a time.

  13. IMPLEMENT Previous work on implementation: 1998 Almeida, Dabu, Manikutty, Cao: Prioritizing HTTP requests at Web servers. “Nice” the low-priority process. 1999 Crovella, Frangioso, Harchol-Balter: SRPT scheduling of requests. User-level scheduling of reads and writes. What is the problem with both of these?

  14. Web Server Network/O.S. insides of traditional Web server IMPLEMENT Socket 1 Client1 Network Card Socket 2 Client2 BOTTLENECK Client3 Socket 3 Sockets take turns draining --- FAIR = PS.

  15. IMPLEMENT Web Server Network/O.S. insides of our improved Web server Socket 1 Client1 S Network Card 1st Socket 2 Client2 2nd M BOTTLENECK 3rd Client3 Socket 3 L priority queues. Socket corresponding to file with smallest remaining data gets to feed first.

  16. IMPLEMENT Experimental Setup 1 1 APACHE WEB SERVER CLIENTS 2 2 3 3 Direct Connection 10Mb/s 200 200 Linux 0.S. Linux 0.S. • Implement SRPT-based scheduling: • 1) Modifications to Linux O.S.: 6 priority Levels • 2) Modifications to Apache Web server • 3) Priority algorithm design.

  17. IMPLEMENT Experimental Setup 1 1 APACHE WEB SERVER 2 2 CLIENTS 3 3 200 200 Linux 0.S. Linux 0.S. • Comparison of SRPT implementation vs. FAIR scheduling standard. • Comparison done under both APACHE and FLASH web servers. • Comparisons done under trace-based workload and under • Web workload generator. • Under trace-based workload: • -- Number requests made: 1,000,000 • -- Size of file requested: 41B -- 2 MB • -- Distribution of file sizes requested has HT property. • Experiment under range of loads.

  18. IMPLEMENT Preliminary Comments to Results: 1 1 APACHE WEB SERVER 2 2 CLIENTS 3 3 200 200 Linux 0.S. Linux 0.S. • Measured job throughput, byte throughput, and bandwidth • utilization were same under SRPT and FAIR scheduling. • Same set of requests complete. • No additional CPU overhead under SRPT scheduling. • CPU utilization always 1% - 5%. • Network was bottleneck in all experiments.

  19. Results: Mean Response Time Mean Response Time (ms) FAIR SRPT Load

  20. Results: Mean Slowdown FAIR Mean Slowdown SRPT Load

  21. Results: Mean Response Time vs. Size Load = 0.8 Mean Response Time (ms) FAIR SRPT Requested File Size (bytes)

  22. Mean Response Time vs. Size Percentile Load =0.8 FAIR Mean Response time (ms) SRPT Percentile of Request Size

  23. Summary so far ... • SRPT scheduling yields significant improvements in Mean Response Time at the server. • Negligible starvation. • No CPU overhead. • No drop in throughput.

  24. More questions … • This study involved a LAN. • Are the effects ofSRPT in a WANas strong? • So far we’ve only experimented with load < 1. • What happens under SRPT vs. FAIR when the • server runs under transient overload? • -> new analysis • -> implementation study

  25. WAN setting Question: Will improvement of SRPT over FAIR scheduling appear greater in a LAN setting or a WAN setting?

  26. WAN setting Answer: Normally LAN setting shows more improvement. But for very high load, WAN setting shows more improvement. Why? Load 0.7 Load 0.9

  27. Zzzzzzz zzz... Overload Person under overload

  28. Web server under overload When reach SYN-queue limit, server drops all connection requests. SYN-queue Clients Server SYN-queue ACK-queue Apache-processes

  29. FAIR vs. SRPT Question: What will happen under FAIR vs. SRPT wrt number of connections at server and wrt response times? (a) Given persistent overload (b) Given transient overload r>1 r>1 r>1 r>1 r<1 r<1

  30. Persistent Overload: Time until SYN-queue limit is hit r=1.5 FAIR FAIR+ (tuned version) SRPT

  31. Persistent Overload: Buildup in number of connections + FAIR+ SRPT

  32. Transient Overload: Buildup of Connections at Server FAIR+ SRPT

  33. Transient Overload: Mean response time (in msec) FAIR+ SRPT

  34. Transient overload: Response time as function of job size FAIR+ SRPT small jobs win big! big jobs aren’t hurt! WHY?

  35. Conclusion • SRPT scheduling yields significant improvements in Mean Response Time at the server under LAN, WAN, under high load, and under overload. • Negligible or zero unfairness. Often better for all requests. • All results corroborated via implementation and analysis.

More Related