On the Scale and Performance of Cooperative Web Proxy Caching

1. On the Scale and Performance of Cooperative Web Proxy Caching University of Washington Alec Wolman, Geoff Voelker, Nitin Sharma, Neal Cardwell, Anna Karlin, Henry Levy

2. Hits Internet Misses Misses Clients ProxyCache Servers Caching for a Better Web • Performance is a major concern in the Web • Proxy caching is the most commonly used method to improve Web performance • Duplicate requests to the same document served from cache • Hits reduce latency, network utilization, server load • Misses increase latency (extra hops)

3. Cache Effectiveness • Previous work has shown that hit rate increases with population size [ Duska et al. 97, Breslau et al. 98 ] • A single proxy cache has practical limits • Load, network topology, organizational constraints • One technique to scale the client population is to have proxy caches cooperate

4. Proxy Clients Internet Clients Clients Cooperative Web Proxy Caching • Sharing and/or coordination of cache state among multiple Web proxy cache nodes • Effectiveness of proxy cooperation depends on: • Inter-proxy communication distance • Size of client population served • Proxy utilization and load balance

5. Cooperative Web Caching • How much benefit does cooperative caching provide in the Web environment?

6. Outline • Introduction & related work • Trace methodology • Cooperative caching for small & medium scale populations • Scaling to larger client populations • Latency • Conclusions

7. Previous Research • Cooperative proxy caching is a popular research topic: [e.g. Chankhunthod et al. 96, Zhang et al. 97, Fan et al. 98, Krishnan et al. 98, Menaud et al. 98, Tewari et al. 98, Touch 98, Karger et al. 99 ...] • Focus is on highly scalable algorithms • Some seek to scale to the entire Web

8. Challenges • No real understanding of document sharing across diverse organizations • Little analytic or empirical evaluation of these algorithms using realistic workloads for large-scale client populations • Problem: • Evaluating cooperative proxy caching requires multiple simultaneous traces of Web proxies, across a diverse set of organizations

9. Our Contribution • We use multi-organization traces to evaluate cooperative proxy caching at small and medium scales • We use analytic modelling to evaluate cooperative caching at scales beyond those available in our traces

10. A Multi-Organization Trace • University of Washington (UW) is a large and diverse client population • Approximately 50K people • UW client population contains 200 independent campus organizations • Museums of Art and Natural History • Schools of Medicine, Dentistry, Nursing • Departments of Computer Science, History, and Music • A trace of UW is effectively a simultaneous trace of 200 diverse client organizations

11. Cooperation Across Organizations • By considering each UW organization as an independent “company” with its own clients and its own proxy, we can empirically evaluate cooperative caching across diverse client populations • How much Web document reuse is there between these organizations? • Place a proxy cache in front of each organization. What is the benefit of cooperative caching among these 200 proxies?

12. UW Trace Characteristics • Trace collected at UW network border (May 1999) • Filtered: requests from UW clients, responses from external Web servers • Most of requests come directly from clients (0.5% come from proxies)

13. Question • What is the benefit of cooperative caching among the 200 UW organizational proxies?

14. Ideal Hit Rates for UW proxies • Ideal hit rate - infinite storage, ignore cacheability, expirations • Average ideal localhit rate: 43%

15. Ideal Hit Rates for UW proxies • Ideal hit rate - infinite storage, ignore cacheability, expirations • Average ideal localhit rate: 43% • Explore benefits of perfect cooperation rather than a particular algorithm • Average ideal hit rate increases from 43% to 69% with cooperative caching

16. Cacheable Hit Rates forUW proxies • Cacheable hit rate - same as ideal, but doesn’t ignore cacheability • Cacheable hit rates are much lower than ideal (average is 20%) • Average cacheable hit rate increases from 20% to 41% with (perfect) cooperative caching

17. Scaling Cooperative Caching • Organizations of this size can benefit significantly from cooperative caching • We don’t need cooperative caching to handle the entire UW population size • A single proxy (or small cluster) can handle this entire population! • No technical reason to use cooperative caching for this environment • In the real world, decisions of proxy placement are often political or geographical • How effective is cooperative caching at scales where a single cache will not work?

18. Hit Rate vs. Client Population • Curves similar to other studies • [e.g., Duska97, Breslau98] • Small organizations • Significant increase in hit rate as client population increases • The reason why cooperative caching is effective for UW • Large organizations • Marginal increase in hit rate as client population increases

19. Extrapolation to Larger Client Populations • Use least squares fit to create a linear extrapolation of hit rates • Hit rate increases logarithmically with client population, e.g., to increase hit rate by 10%: • Need 8 UWs (ideal) • Need 11 UWs (cacheable) • “Low ceiling”, though: • 100% at 11.3M clients (UW ideal) • 61% at 2.1M clients (UW cacheable) • A city-wide cooperative cache would get all the benefit

20. Question • What is the benefit of cooperative caching among large organizations?

21. UW & Microsoft Cooperation • What if we ran a wire across Lake Washington, to connect UW & Microsoft? • We collected a Microsoft proxy trace during same time period as the UW trace • Combined population is ~80K clients • Increases the UW population by a factor of 3.6 • Increases the MS population by a factor of 1.4

22. UW & Microsoft Traces

23. UW & MS Cooperative Caching • Is this worth it?

24. What about Latency? • From the client’s perspective, latency matters far more than hitrate • How does latency change with population? • Median latencies improve only a few 100 ms with ideal caching compared to no caching. • On average, a web page consists of 4.5 HTTP objects

25. Conclusions • A negative result: without significant workload changes, designing highly-scalable cooperative proxy-cache schemes is unnecessary • Largest benefit is achieved with small populations (up to 2K-5K clients) • Limited benefit of cooperation when we combined the UW & Microsoft populations • Document cacheability is a severe limitation with current workloads • Analytic model results: • Confirm that most of benefit is obtained once you reach populations the size of a large city • Future workloads: large-scale cooperative caching could become more relevant with different rate-of-change characteristics

26. UW Cooperative Caching Results

27. Extrapolating UW & MS Hit Rates

28. UW Latency

29. Complete Hit Rate Graph

30. Blank Slide • blankness here...