1 / 42

A Case Study of Web Server Benchmarking Using Parallel WAN Emulation

A Case Study of Web Server Benchmarking Using Parallel WAN Emulation. Carey Williamson Rob Simmonds Martin Arlitt University of Calgary. Network Emulation. A hybrid performance evaluation methodology that combines aspects of implementation with simulation modeling

milesk
Download Presentation

A Case Study of Web Server Benchmarking Using Parallel WAN Emulation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Case Study ofWeb Server Benchmarking UsingParallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary

  2. Network Emulation • A hybrid performance evaluation methodology that combines aspects of implementation with simulation modeling • A network emulator is a network simulator with an interface that allows client applications to interact with it in real-time • “A simulator that talks back” (IP packets) • Why? Provides a reliable, repeatable test environment for distributed applications • Internet games, video conferencing, Web, ... Web Benchmarking with IP-TNE

  3. Simulation Real World

  4. IP-TNE • The Internet Protocol Traffic and Network Emulator (IP-TNE) is a network emulator based on IP-TN (packet-level IP network simulator) • Enables interaction between IP based clients via an IP-TN simulated network (in real time!) • Distributed applications can interact with IP-TNE without modification Web Benchmarking with IP-TNE

  5. IP-TNE Overview Web Benchmarking with IP-TNE

  6. IP-TNE Overview (cont’d) • CCTKit withreal-time extensions provides an environment for fast network emulation (PDES) • IP-TNE provides routing methods suitable for shared environments and dedicated test environments • Now has HTTP and TCP client models that can be used for Web server benchmarking Web Benchmarking with IP-TNE

  7. So What? • Flexible routing model support • High-performance packet reading and writing via raw sockets (1 Gbps) • Can model an arbitrary IP internetwork • Detailed IP protocol models • IPv4, ICMP, ping, traceroute, pchar, MTU, ... • Supports parallel execution on shared memory multiprocessors • “Blazingly fast!” - CLW, 2002 Web Benchmarking with IP-TNE

  8. Example: Web Benchmarking Client 1 Client 2 Client 3 Web Server ... Client C Web Benchmarking with IP-TNE

  9. WAN Emulation (1 of 3) Client 1 Client 2 Client 3 Web Server ... “Centralized” Approach Client C Web Benchmarking with IP-TNE

  10. WAN Emulation (2 of 3) Client 1 Client 2 Client 3 Web Server ... “Shim” Approach (NISTnet, DummyNet, WASP) Client C Web Benchmarking with IP-TNE

  11. WAN Emulation (3 of 3) Client 1 Client 2 Client 3 Web Server ... Client C Our IP-TNE Approach Web Benchmarking with IP-TNE

  12. Objectives of Case Study • Evaluate new approach to WAN emulation, and demonstrate feasibility • Confirm prior results by Nahum et al. on effects of WAN conditions on Web server performance • How fast can Apache Web server go? • How fast can IP-TNE go? Web Benchmarking with IP-TNE

  13. Experimental Setup • IP-TNE on Compaq ES-40 (4 CPU) • Apache (1.3.23) on another ES-40 • Gigabit Ethernet (1 Gbps) in between • OS is Compaq Tru64 (v5.1A) • ANML for defining network model • e.g., simple regular WAN topology Web Benchmarking with IP-TNE

  14. Web Benchmarking with IP-TNE

  15. Web Benchmarking with IP-TNE

  16. Web Workload Model • Static content only • Closed-loop workload generator • Fixed-size Web objects • Small (1 KB) • Large (64 KB) • Variable-size Web objects • Median 3 KB • Mean 9 KB • Pareto heavy tail (alpha = 1.2) • Zipf-like document popularity profile Web Benchmarking with IP-TNE

  17. Performance Metrics • Two primary metrics • HTTP transaction rate (trans/sec) • Network throughput (Mbps) • Several secondary metrics • Response time • Connection failure rate • Packet loss rate • ... Web Benchmarking with IP-TNE

  18. Results (Fig. 4a) • For 1 KB transfers with HTTP/1.0: • Single client: 170 transactions/sec • Transaction rate scales up with number of clients up to about H = 32 • Transaction rate flattens, then drops sharply as num clients is increased more (closed loop) • Peak rate achieved: 3800 trans/sec • Peak throughput approximately 40 Mbps • Transaction rate is (strongly) inversely related to the client round trip time (RTT) Web Benchmarking with IP-TNE

  19. Web Benchmarking with IP-TNE

  20. Web Benchmarking with IP-TNE

  21. Web Benchmarking with IP-TNE

  22. Web Benchmarking with IP-TNE

  23. Web Benchmarking with IP-TNE

  24. Results (Fig. 4b) • For 64 KB transfers with HTTP/1.0: • Single client: 18 transactions/sec • Transaction rate scales up with number of clients up to about H = 32 • Transaction rate flattens, then drops slightly as num clients is increased more • Peak rate achieved: 220 trans/sec • Peak throughput approximately 115 Mbps • Transaction rate is (weakly) inversely related to the client round trip time (RTT) Web Benchmarking with IP-TNE

  25. Web Benchmarking with IP-TNE

  26. Results (Fig. 4c) • For variable-size transfers with HTTP/1.0: • Single client: 60 transactions/sec • Transaction rate scales up with number of clients up to about H = 32 • Transaction rate flattens, then drops as num clients is increased more • Peak rate achieved: 1300 trans/sec • Peak throughput approximately 90 Mbps • Transaction rate is inversely related to the client round trip time (RTT) • Behaviour is in between 1 KB and 64 KB results Web Benchmarking with IP-TNE

  27. Web Benchmarking with IP-TNE

  28. Results (Fig. 5a) • Concurrent connections with HTTP/1.0: • Single client: 600 transactions/sec • Qualitatively similar results to before, except that fewer clients are needed to drive the server to full load • Conceptually concurrent connections are no different than adding more clients Web Benchmarking with IP-TNE

  29. Web Benchmarking with IP-TNE

  30. Results (Fig. 5b) • Persistent connections with HTTP/1.1: • Single client: 300 transactions/sec • Qualitatively similar results to before, except that transaction rate is about 70% higher than for HTTP/1.0 (since multiple HTTP req’s per TCP conn) • Peak transaction rate 6500 trans/sec • Much less dependency on RTT effects Web Benchmarking with IP-TNE

  31. Web Benchmarking with IP-TNE

  32. Results (Fig. 5c) • Pipelined persistent connections with HTTP/1.1: • Single client: 800 transactions/sec • Qualitatively similar results to before, except that transaction rate is about 100% higher than for HTTP/1.0 • Peak transaction rate 7600 trans/sec • Much less dependency on RTT effects Web Benchmarking with IP-TNE

  33. Web Benchmarking with IP-TNE

  34. Results (Fig. 6a) • Effect of WAN RTT delays: • Increasing the per-link propagation delay increases the client RTT delay, which in turn reduces the transaction rate and throughput (as expected) • As RTT increases, more and more clients are needed in order to drive the Web server to full load • Similar to [Nahum et al. 2001] Web Benchmarking with IP-TNE

  35. Web Benchmarking with IP-TNE

  36. Results (Fig. 6b) • Effect of bandwidth asymmetry: • For asymmetric access technologies such as ADSL (Asymmetric Digital Subscriber Line), the upstream link from the client to the server can sometimes be the bottleneck for TCP, even though it is primarily carrying ACKs only • Depends on normalized bandwidth ratio • Greater asymmetry, worse performance Web Benchmarking with IP-TNE

  37. Web Benchmarking with IP-TNE

  38. Results (Fig. 6c) • Effect of WAN packet losses: • Decreasing the router queue size at the bottleneck link increases the packet loss ratio (as expected) • As the level of packet loss increases, the HTTP transaction rate and the network throughput decrease • Similar results to [Nahum et al. 2001] Web Benchmarking with IP-TNE

  39. Web Benchmarking with IP-TNE

  40. Summary and Conclusions • The IP-TNE is a useful tool for Web server benchmarking • Demonstrates feasibility of WAN emulation using a single computer • Confirms prior results by Nahum et al. studying the effects of WAN conditions on Web server performance • Demonstrates performance advantages of HTTP/1.1 Web Benchmarking with IP-TNE

  41. Future Work? • Increase traffic to your Web site!!! • Guarantee 20% increase in traffic! • 1000’s of new clients per month!!!!! • Send $$$ to carey@cpsc.ucalgary.ca Web Benchmarking with IP-TNE

  42. Future Work with IP-TNE • Validation of IP-TNE (and IP-TN) • Benchmarking IP-TNE vs IP-TNE • Benchmarking Web caching appliances • Evaluating SRPT scheduling in WAN setting • Connection/packet-level scheduling algorithms • Evaluating CATNIP approach to TCP/IP • Evaluating portable (wireless) Web servers • Workload sensitivities (Zipf, Pareto, corr, mods) • Experiments with dynamic content (CGI, etc) • Asymmetric networks, Ensemble-TCP • Parallel TCP connections: friend or foe? • Evaluating effect of TCP SACK in WAN Web Benchmarking with IP-TNE

More Related