Optimal Content Delivery with Network Coding

1. Optimal Content Deliverywith Network Coding Derek Leong, Tracey Ho California Institute of Technology Rebecca Cathey BAE Systems • CISS 2009 March 19, 2009

2. Motivation

3. Motivation S

4. Motivation t t t S t t t

5. Motivation t t t S t t t

6. Motivation t t t S t t t

7. Motivation t t t S t t t

8. Motivation t t How to route dissemination and fetch flows? t S t How to pick storage nodes? t t

9. Review: Subgraph Selection

10. Review: Subgraph Selection Resultant shared flow must support individual flows Flows must be realizable within capacity Flows through each node must be conserved

11. Review: Subgraph Selection for CDNs

12. Review: Subgraph Selection for CDNs Storage flow in node memory, occurring over time Network coding allowed Network coding NOT allowed

13. Review: Subgraph Selection for CDNs

14. Review: Subgraph Selection for CDNs Total dissemination cost Total storage cost Total fetch cost

15. Review: Subgraph Selection for CDNs Resultant shared flow must support individual flows Flows must be realizable within capacity Flows through each node must be conserved

16. Review: Subgraph Selection for CDNs

17. Modified Formulations k-hop Fetch Constraint: Restrict fetch flows to within the k-hop neighborhood of each receiver

18. Robust Storage • Distributed storage can be used to improve robustness of data availability in unreliable networks • Intuitively, in a network where nodes or arcs fail probabilistically, the probability of a receiver being able to successfully fetch data increases with the amount of redundant storage and the proximity of storage nodes

19. Robust Storage • Goal: Ensure each receiver can still successfully access content in the event that some nodes or arcs fail during the fetch stage

20. Robust Storage • Exact but prohibitively complexapproach: • Consider all possible failure events = { (occurrence prob, set of nodes & arcs that fail)} • Replace each receiver with a set of virtual receivers, one for each failure event affecting it • Allow fetch flows only through the corresponding unaffected nodes & arcs for each virtual receiver • Modify objective function to include fetch cost incurred by each virtual receiver, weighted by their respective probabilities, so that it continues to express the expected total cost

21. Robust Storage • k-hop fetch constraint reduces the number of virtual receivers required • Example: To protect against the failure of up to any m arcs in the network, the number of virtual receivers required per receiver is • Without fetch constraint : • With 1-hop fetch constraint : • However, the number of virtual receivers still grows exponentially with the number of hops k

22. Robust Storage • To allow flexibility in the number of hops k in the fetch constraint while remaining tractable, we can specify the virtual receivers first, and then compute the resulting minimum fetch success probability • This approach enables us to lower-bound the success probability in terms of • number of hops k • probability of node or arc failure p • number of disjoint paths in the k-hop neighborhood of a receiver d for example

23. Robust Storage • Using the following (d+1) virtual receivers for t • v0 corresponding to the zero-failure event • vi corresponding to the event where only the arcs on paths Pj, j ≠ i, are allowed to carry fetch flows for vi • We can achieve success probability≥ Prob[at most one of the d paths fails]=

24. Other Modified Formulations Potential Storage Nodes: Restrict potential storage nodes to some subset by changing the capacity

25. Other Modified Formulations Storage Budget Constraint: Impose an aggregate storage budget over all nodes

26. Other Modified Formulations Fetch Load Constraint: Bound the expected load on nodes and arcs during the fetch stage

27. Other Modified Formulations Dissemination Stage Receivers: Add node in the dissemination stage to the set of receivers

28. Other Modified Formulations Delivery of Multiple Content: Introduce separate flows for individual content, allowing network coding during only the dissemination stage and among flows for the same content

29. Performance Evaluation • Compare Network-Coded Formulation(NCF)vsMinimum k-median Formulation (KMF),for the Delivery of Multiple Content(“multiobject placement”) subject to agiven storage budget constraint • Major difference between NCF and KMF:KMF minimizes only the total fetch cost, butNCF minimizes total fetch + dissemination cost • Performance metric: = NCF expected total dissemination + fetch cost , KMF expected total dissemination + fetch cost

30. Performance Evaluation • Compare Network-Coded Formulation(NCF)vsMinimum k-median Formulation (KMF),for the Delivery of Multiple Content(“multiobject placement”) subject to agiven storage budget constraint • Major difference between NCF and KMF:KMF minimizes only the total fetch cost, butNCF minimizes total fetch + dissemination cost • Performance metric: Unit {dissemination, fetch} cost = NCF expected total dissemination + fetch cost , KMF expected total dissemination + fetch cost Expected total number of requests by a receiver over all objects

31. Performance Evaluation = 0 (“free dissemination”) NCF KMF

32. Performance Evaluation = 0 (“free dissemination”) NCF KMF Number of objects Storage budget

33. Performance Evaluation = 0 (“free dissemination”) NCF KMF

34. Performance Evaluation Number of objects |W| = 1 NCF KMF

35. Performance Evaluation Storage budget Number of objects |W| = 1 NCF KMF

36. Performance Evaluation Number of objects |W| = 1 NCF KMF

37. Performance Evaluation Number of objects |W| = 3 NCF KMF

38. Performance Evaluation Number of objects |W| = 5 NCF KMF

39. Performance Evaluation Number of objects |W| = 7 NCF KMF

40. Implementation Considerations • Using a domain name system (DNS) to direct origin servers and end users to the nearest available CDN node • Using source routingfor the dissemination stage, and a DNS to resolve requests during the fetch stage • Opportunistically caching coded data flows during the fetch stage

41. Augmenting with a P2P Network • A hybrid CDN–P2P network achieves better scalabilitysince end users help contribute bandwidth, storage, and computation resources • CDN provides a reliable backbonefor content delivery, and prevents severe service degradation in the face of high churn rates • Specifying dissemination stage receivers when adopting a distributed hash table(DHT) based storage and lookup mechanism

42. Conclusion & Future Work • Presented a unified LP formulationfor optimal content delivery in CDNs • Simulation results suggest NCF performs significantly better, even under modest circumstances (small network, few objects, low storage budget, low dissemination costs) • Look forward to addressing content delivery in dynamic environments (e.g. mobile ad hoc networks)

43. Optimal Content Deliverywith Network Coding Derek Leong, Tracey Ho California Institute of Technology Rebecca Cathey BAE Systems • CISS 2009 March 19, 2009