Towards Content Distribution Networks with Latency Guarantees

1. Towards Content Distribution Networks with Latency Guarantees Chengdu Huang and Tarek F. Abdelzaher University of Virginia

2. Outline • Background • Challenges • Contributions • Formulation • Architecture • Evaluation • Conclusion

3. Overview • Our goal is to guarantee a subsecond upper bound on response time • Latency bound formula specified in contract with content provider • Existing research: optimizing average response time • Client perceived latency consists of • Latency from client to a CDN server • Latency from a CDN server to some other CDN server (request forwarding) • Processing time within CDN servers

4. Need and Feasibility-- Observations from the Internet • Establishing the Need: (Latency Analysis) • Average latency for web objects are a significant fraction of a second • A large portion of latencies exceed a second  Bounded-delay CDN is needed • Establishing the Feasibility: (Cost Analysis) • Internet latencies (for a fixed pair) are not time invariant but only oscillate with a small range • Spikes are not very common • Can be attributed to underutilization of Internet backbone  Replica locations are relatively static – maintenance cost is low

5. Contributions • Mapping the latency bound guarantee problem to a well-studied graph theoretic problem • Designed and implemented a real-time CDN system on a WAN platform • Extensive evaluation results drawn from an Internet deployment of the service prototype

6. Mapping • The problem of achieving latency bound is mapped to a graph domination problem • Formulation • Given a set of CDN servers S= {S1,…, Sn}, a content object C, and its latency bound L • Construct a graph G whose vertices are S • Edge SiSj is added to G iff server Si can download C from server Sj within a time less than L • To find minimal dominating set D: a subset of S with minimal cardinality that for all u in S - D, there is a v in D for which uv is in G • Nodes represent servers, edges connect neighbors reachable within latency bound  dominating set is reachable within latency bound from any server

7. B G E C F A D Mapping

8. Mapping B G E C F A D Graph Domination Problem

9. Existing Graph Domination Algorithms • Centralized greedy heuristic • Repeatedly selects the vertex with highest remaining degree • Best approximation known • Distributed algorithms • DDCH (INFOCOM’00) • LRG (PODC’01) • Kuhn and Wattenhofer (PODC’03) • Limitations • Performance in asynchronous environment • Need multiple rounds to finish: long termination time • We developed a new distributed, asynchronous algorithm

10. Architectural Challenges • The CDN system runs in a highly dynamic and asynchronous environment • How to handle content objects with different sizes • Absence of global knowledge

11. Challenge: Asynchronous environment • Our distributed algorithm • Goal: Decentralized, asynchronous, fast termination • Idea • Inspired by the centralized counterpart • Nodes independently nominate the neighbor with the highest degree • Receiving nomination makes a node join the dominating set and send out dominator announcement • Receiving dominator announcement makes a node refrain from sending nomination • Insights: • High degree nodes quickly join the dominating set • Joining of high degree nodes quickly inhibits further nominations

12. E: 3 F: 3 G: 3 C: 5 E: 3 F: 3 G: 3 A: 4 B: 3 C: 5 A: 4 B: 3 C: 5 D: 2 E: 3 Degree=3 Degree=3 Degree=3 A is reachable Degree=5 A is reachable A: 4 B: 3 C: 5 D: 2 Degree=3 A: 4 C: 5 D: 3 E: 3 F: 3 G: 3 Degree=4 A is reachable Degree=2 A, B, D, E send NOMINATION to C F, G send NOMINATION to E (random tie-breaking) Algorithm -- example B G E C F A Reachable D

13. Degree=3 Degree=3 Degree=3 Degree=5 Degree=3 Degree=4 Degree=2 Mapping B G E C F A D

14. Challenge: probing objects of different sizes • Probing is needed to estimated latency • Latency depends on file size which can be any size, making probing challenging • Solution • Probe a series objects of certain sizes • Assuming latency has a simple linear relation with object size • Use a recursive least square (RLS) estimator to estimate the parameters and • More sophisticated probing techniques can be plugged in

15. Challenge: objects of different sizes • Validation of our latency-size model

16. Challenge: absence of global knowledge • The system should perform well without global knowledge • Introduce a parameter: visibility • Percentage of servers in the system each server knows when it starts • Low visibility incurs more replicas • Two heuristics to reduce number of replicas • Reciprocal mode • Highest degree node exchange

17. Implementation • Instrument Squid Proxy Cache • Deployed on PlanetLab • PlanetLab is a WAN platform with more than 100 sites across 20+ countries • Deployed on 30~80 nodes

18. Experiment on PlanetLab

19. Evaluation outline • Efficiency • Latency bound guarantee • Absence of global knowledge

20. Evaluation • Number of replicas Latency Bound:

21. 30 Nodes 80 Nodes Evaluation • Termination Time

22. Evaluation • Latency bound guarantee • Baselines • Single Server • Random (with the same number of replicas) • Average Latency Greedy (Qiu INFOCOM’00)

23. Evaluation • Latency bound guarantee

24. Latency Bound Guarantee • Number of Replicas Evaluation • Absence of global knowledge

25. Conclusion • Designed and implemented a CDN system that provides latency bound • Based on a distributed algorithm that performs well in asynchronous environment • Experiment results show that latency bound can be achieved with a very high confidence