On the Scale and Performance of Cooperative Web Caching

1. On the Scale and Performance of Cooperative Web Caching Alec Wolman, et al. Presented by: Christopher Banek & William Conner CS598ig November 4, 2004

2. Basics of Web Caching • Web caching is the storage of downloaded content (HTML, images, etc) for using to service requests to the same content • Why do we cache web content? • Reduce latency to fetch content • Reduce bandwidth use and network congestion by not downloading the same content multiple times

3. Basics of Web Caching • What are the current problems with web caching? • Not all data is cacheable (streaming audio/video, dynamic data, private data, cookies) • With a finitely sized storage device, what do we choose to store? • How can we increase the hit rate? • What models can we use to model web traffic? (Zipf’s law?)

4. Hit rate as a function of population size • Let N be the number of users using the cache • N = 1 is a local browser cache, most things you’ve viewed recently is stored on the local harddisk • What if N > 1? • Most papers find a natural logarithmic relation to hit rate and population size

5. Case Study: Microsoft Corporation & University of Washington • Information about the traces: • Gathered at UW (~50,000 students) and Microsoft (~40,000 employees) simultaneously over a week in May 1999 • Anonymized in the same way to allow more effective correlation • In total, around 190 million HTTP requests

6. Case Study: Microsoft Corporation & University of Washington Notice knee of curve at around N = 2500

7. Latency as a function of population size

8. Other factors as a function of population size

9. Cooperative Caching • Pass on cache misses to another cache hoping they will have the content • Current Problems with Cooperative Caching • Who do you pass the miss to? • In this paper, assume ideal cooperative caching and see how good the results could be.

10. Cooperative caching in small populations • Slope of hit rate vs population is greatest at smaller populations • This means combining many small (N < 2500) groups should show a much higher cache hit rate • Let’s look at some data from the University of Washington, with their different departments each representing a different small group

11. University of Washington Departmental Caching Study

12. Cooperative caching in large populations • Since we get a big benefit from small groups, what about large groups? • Combining the two groups gives the following population increases • Microsoft, N = 60,000 to 83,000 (1.4x) • University of Washington, N = 23,000 to 83,000 (3.6x)

13. Cooperative caching in large populations • UW Ideal hit rate increases 5.1% • UW Cacheable rate increases 4.2% • MS Ideal hit rate increases 2.7% • MS Cacheable rate increases 2.1%

14. Cooperative caching in large populations • Why is the increase so low? • Documents that are popular tend to be Universally popular, and in all of the cooperating caches • Documents that are not popular tend to be in none of the caches, and therefore are always misses

15. A model for cooperative caching • N clients in the population • n documents that can be cached • λis the average client request rate. • μis the average rate of change of a document • p is the probability of a document being cacheable • E(S) is the average size of a document • E(L) is the average latency of a document

16. A model for cooperative caching • Steady-state hit rate: • Where C is the probability that a request is a hit and defined as: • The latency being: • Bandwidth savings due to caching:

17. A model for cooperative caching • Parameters that model the University of Washington trace

18. A model for cooperative caching • The importance of μ in the model:

19. Applications of the model • Generate models of three different types of caches, and see how they perform in different populations: • A medium sized city (N = .5 million) • A small state (N = 5 million) • The West Coast (N = 50 million)

20. Applications of the model • Equations representing different cooperative caching schemes:

21. Applications of the model

22. Conclusions • The benefits from highly scalable and intelligent cooperative caching solutions diminish for very large populations (N > population of a small city), but work very well for small populations and groups • Largest benefits for cooperative caching come by cooperating between many small groups • Performance of cooperative caches (and caches in general) are limited by the cacheability of documents • Grouping users together by mutual interest offers little performance gain over randomly assigned groups

23. Discussion Topics • Many cooperative caching schemes still being devised despite these results • These results are shown from the client side, how are content servers affected by cooperative caching? • For small groups, when would cooperative caching be used, and when would it be better to use a larger monolithic cache system?

24. Caching • Papers • The case for cooperative networking • On the scale and performance of cooperative web proxy caching • Presentation • Caching technologies for web applications Presenters: William Conner & Chris Banek

25. The case for cooperative networking Venkata N. Padmanabhan Kunwadee Sripanidkulchai

26. CoopNet • End-hosts cooperate to improve network performance perceived by all • Complements traditional client-server communication • Only kicks in when needed • Clients share previously downloaded content with others during flash crowds

27. Flash Crowds • Originally used in science fiction short story • Thousands travel back in time to see historical events via teleportation • In computer science, refers to rapid and dramatic surge in volume of requests • Also referred to as slash dot effect • Often servers become overwhelmed and response times increase dramatically • Used traces from MSNBC site on 9/11

28. Flash Crowds • Triggered by events of great interest • Planned • e.g., sport events, music concerts • Unplanned • e.g., earthquakes, plane crashed • Even modest flash crowd can overwhelm a server with low capacity • Shares some characteristics with DoS attacks but flash crowd requests are legitimate

29. Bottleneck during Flash Crowds • Probably not disk bandwidth • Relatively small number of popular files could fit into server’s main memory buffer cache • 0.37% of files accounted for 90% of accesses at MSNBC web site during 9/11 • 2.87% of files accounted for 99% of accesses at MSNBC web site during 9/11

30. Bottleneck during Flash Crowds • Probably not CPU • For MSNBC, bottleneck at first with dynamically generated content • MSNBC switched to static HTML • Error status code responses dropped from 49.4% to 6.7%

31. Bottleneck during Flash Crowds • Network bandwidth is the cause • Highly tuned Web server could have throughput of 1 Gbps • Study by UW showed bandwidth was less than 1.5 Mbps for 65% of servers and less than 10 Mbps for 90% of servers • Likely that links close to server are worst affected by flash crowd

32. Basic Design • Server redirects HTTP requests from clients to other clients during flash crowd • Some or all requests redirected • Clients resend request to one or more peers that have downloaded URL in recent past • Alleviates load on server

33. Basic Design

34. Basic Design • Clients indicate willingness to participate with HTTP pragma field in request header • Server remembers IP addresses of files that have requested each file in recent past • List constantly updated to spread load • Redirection messages contain between 5 and 50 addresses • 200-300 bytes • Slimmed-down MSNBC front page 18-22 Kbytes

35. Basic Design • Prototype implementation • Server implemented as an extension to Microsoft Information Internet Services server • Client implemented as a client-side proxy that serves requests from both local browser and peers

36. Peer Selection • Topological closeness • Server provides addresses that are topologically close based on common IP address prefixes • Information derived from BGP routing tables • If no matching prefix, random list provided

37. Peer Selection • Topological closeness • Problem • Matching prefix does not always mean topological closeness • e.g., National or global ISPs • Solution (Quick Check) • Each peer measures delay to small number of landmark hosts • Peers with similar delay coordinates are close to each other

38. Peer Selection • Network bandwidth • Clients report bandwidth to server as part of requests • Estimated by passive monitoring of network traffic • Server includes peers with similar bandwidth in redirection messages • Low-bandwidth clients are limited anyway • No incentive to under-report

39. Peer Selection • Client chooses peers from redirection message • Select fastest connections • Request non-overlapping chunks of file from multiple peers • Acts as an additional test to find fast download • Especially useful for streaming large media files because the burden on individual peers is reduced

40. Redirection Revisited • Server may still be overwhelmed • If large number of redirection messages • If only small number of clients join CoopNet • Participating clients check with peers first • Scalable, distributed search might be unnecessary for small number of clients • Proposed peer groups with root nodes • Root nodes assigned based on hash of URL • Root node keeps pointers to all copies of URL in group

41. Security • Ensuring integrity • Server digitally signs content • Client obtains signatures from server or peer • Ensuring privacy • No solution proposed • Downloaded content is probably of widespread interest anyway

42. Experiments • Simulations based on traces collected at MSNBC during 9/11 flash crowd • 1 hour with 40 million requests • Participating client initially contacts server to get IP addresses of peers • Server places 5 to 50 peers in redirection messages • Client contacts peers first and only contact server if content not found at peers

43. Experiments • New hit rate: fraction of requests for new files that can be served by hosts in peer group • Fresh hit rate: fraction of requests for previously downloaded file where a fresher copy can be served by hosts in a peer group

44. Experiments number of CoopNet clients = 200

45. Experiment peers idle 80% of time peer list size = 30

46. Conclusion • CoopNet can reduce load on servers and latency experienced by clients • Solution is too centralized if clients do not contact peers first • No solution presented to new problem of causing flash crowd at peers (future work)

47. Discussion • Is a DHT-based scheme (like Chord) really too heavyweight for CoopNet? • Global events that have a long duration • Clients repeatedly requesting updates • Usefulness during DoS attack?

48. Caching technologies for web applications C. Mohan Some slides taken directly from C. Mohan’s presentation

49. Motivation • Caching improves performance • Web proxy servers • Processor caches in hardware • Buffer pools in DBMSs • Caching for e-commerce transactions is focus of Mohan’s presentation

50. Caching Considerations • Location of cache • Client, proxy, ISP, Web server, DBMS • Granularity of caching • Web page, page fragment, SQL query result • Caching and invalidation policies