Network Coding for Large Scale Content Distribution

1. Network Coding for Large Scale Content Distribution Pablo Rodriguez Microsoft Research Christos Gkantsidis Georgia Institute of Technology IEEE INFOCOM 2005 Presented by Ryan

2. Outline • Introduction • Related Works • Model for Cooperative Content Distribution • Performance Evaluation • Conclusion and Future Works

3. Introduction • Large Scale Content Distribution • Typical content distribution solutions • CDN – Content Delivery Network • Placing dedicated equipment around the network • e.g. Akamai • Cooperative content distribution solutions • Self-scalable • Preventing sudden surge of traffic to the source • e.g. BitTorrent

4. Introduction • Network Coding • Allowing intermediate nodes to encode packets • Making optimal use of the available network resources

5. Introduction • An example • Without a global coordinated scheduler • Node B, receiving Packet 1 or 2 from Node A?

6. Introduction • Contributions in the Paper • Proposing a practical system based on network coding • Not require the knowledge of the underlying topology and centralized scheduling • Robust to extreme situations with sudden server and nodes departures • Better performance comparing to source coding and no encoding schemes

7. Related Works • Tree-Based Cooperative Systems • Creating and maintaining shortest-path multicast trees • Bandwidth-limited (by the bottleneck link on the path from the server) • e.g. SplitStream

8. Related Works • Mesh Cooperative Architectures • Improving the download rates by using parallel downloads • Under-utilizing the network resources (the same block traveling over multiple competing paths) • e.g. BitTorrent

9. Related Works • Erasure Codes • Reconstructing the original content of size n from roughly a subset of any n symbols from a large universe of encoded symbols • Network Coding • Based on theoretical calculations (with the detailed knowledge of the topology and a centralized scheduler)

10. The Model • Server • Dividing the file into k blocks • Uploading blocks at random to different clients • Clients (Users) • Collaborating with each other to assemble the blocks and reconstruct the original file • Exchanging information and data with only a small subset of others (neighbors) • Symmetric neighborhood and links

11. The Model • Upon arrival • Contacting a centralized server (like the tracker in BitTorrent) to get a random list of users in the system • Connecting to the returned users to construct the neighborhood

12. The Model • Content Propagation • 1) No Coding • 2) Source Coding • 3) Network Coding

13. The Model • No Coding and Source Coding • Based only on local information for deciding which block to transfer • Random • A random block • Local Rarest • The rarest block in the neighborhood

14. The Model • e.g. BitTorrent system • A combination of the Random and Local Rarest schemes • Random for the first few blocks • Local Rarest afterwards

15. The Model • Network Coding • The node generates and sends a linear combination of all the information available to it

16. The Model • Recovering the original file after receiving k blocks (associated coefficient vectors are linearly independent to each other) • Just solving the system of linear equations

17. The Model • Incentive Mechanisms • Discouraging free-riding • Scheme 1 • Preference to mutual exchanges • Scheme 2 (Tit-for-tat) • Bounding the absolute difference of uploading minus downloading from one to another

18. Performance Evaluation • Round based simulator • Input • Overlay topology • Users’ upload and download capacities • Server’s capacity • Capacity: number of blocks that can be downloaded/uploaded in a single round • Size of file to distribute • Metric • Download finish time

19. Performance Evaluation • Connecting to 4 peers when joining • Max number of neighbors = 6 • Discovering new neighbors when the utilization of the download capacity is below a certain threshold (10%)

20. Performance Evaluation • Homogeneous topologies • 200 users with capacity = 1 • Server’s capacity = 1 • File size = 100 blocks No Coding Source Coding Network Coding

21. Performance Evaluation • Topologies with clusters • Two clusters, 100 users each • Capacity • Within cluster = 8 • Cluster to cluster = 4 • Server • Capacity = 4 • Departing at round 30 • File size = 100 blocks

22. Performance Evaluation No Coding Source Coding Network Coding

23. Performance Evaluation • Heterogeneous capacities • 10 fast users with capacity = 4 • 190 slow users with capacity = 1 • Server’s capacity = 4 • File size = 400 blocks No Coding Source Coding Network Coding

24. Performance Evaluation • Minimum finish time for the fast users = 50 rounds

25. Performance Evaluation • Dynamic Arrivals • 40 empty nodes every 20 rounds • Capacity = 1 • Staying in the system 10 more rounds after finishing • Server’s capacity = 1 • File size = 100 blocks

26. Performance Evaluation

27. Performance Evaluation • Robustness to node departures

28. Performance Evaluation • Leaving after serving 5% extra blocks • Network coding : 100% finish • Source coding : 40% finish • No coding : 10% finish Network Coding Source Coding No Coding

29. Performance Evaluation • Incentive mechanisms • Max difference = 2 (tit-for-tat)

30. Conclusion • A new content distribution system • Not require knowledge of the whole network topology • Easy to schedule content propagation • Good performance in simulations • Download finish time • Robust to server and users departures • Avalanche – a real system implementation using network coding

31. Future Works • Speed of encoding and decoding • Encoding : O(k) • Decoding : inverting a matrix O(k3), reconstructing the file O(k2) • Dominated by reconstruction • Many reads of large blocks from the harddisk • Protection against malicious nodes • Introducing arbitrary blocks • Making the reconstruction of the original file impossible

32. THANK YOU