ROMA: Reliable Overlay Multicast with Loosely Coupled TCP Connections

1. ROMA: Reliable Overlay Multicast with Loosely Coupled TCP Connections Gu-In Kwon and John Byers Computer Science Dept. Boston University

2. IP Multicast • Highly efficient • Challenges • Congestion Control • Reliability MIT Berkeley UCSD CMU routers end systems multicast flow http://esm.cs.cmu.edu/Sigcomm2001/SigcommTalk.ppt

3. Overlay Multicast MIT1 MIT Berkeley MIT2 UCSD CMU1 CMU CMU2 Berkeley MIT1 Overlay Tree MIT2 UCSD CMU1 CMU2 • Traditional Performance Metrics • Stretch • Relative Delay Penalty • Stress • # of identical packet over a physical link http://esm.cs.cmu.edu/Sigcomm2001/SigcommTalk.ppt

4. Potential Benefits over IP Multicast MIT1 MIT Berkeley MIT2 UCSD CMU1 CMU CMU2 • Quick deployment • ISP’s reluctant to turn on IP Multicast • All multicast state in end systems • Routers maintain per-group state in IP Multicast • Congestion control easier on “unicast” end to end connections http://esm.cs.cmu.edu/Sigcomm2001/SigcommTalk.ppt

5. Past Work on Overlay Multicast • Yoid, Narada, Scattercast, Overcast, NICE, ALMI, RMX, PRM, ZIGZAG, OMNI ……. • Target Application • Non-reliable Streaming Application • Design Goal • Low overhead on tree construction. • Optimize the performance metrics. • Stretch and Stress

6. Reliable Content Delivery C B A 5Mbps 2Mbps Available Bandwidth • Claim: • TCP on each overlay link is sufficient for reliable delivery.

7. Store-and-forward approach A S • Back-pressure mechanism • If the application buffer is full, ask the parent to reduce the sending rate to avoid the buffer overflow. • ALMI(USITS’01) and MCC(NGC’02). Single rate Multicast Congestion Control 1Mbps 10Mbps 10Mbps 6Mbps 1Mbps 1Mbps B C 1Mbps 1Mbps 8Mbps D E F

8. ROMA Contributions • ROMA • Reliable Overlay Multicast Architecture. • Multirate. • TCP on each overlay link. • Forward-when-feasible. • Digital Fountain Approach. • Performance evaluation of the chains of TCP. • Loosely coupled TCP connections. • Conventional wisdom on overlay network is not correct. • Conventional wisdom: Increased latency and loss rate will reduce the performance of overlay node comparing to the direct unicast.

9. Forward-when-feasible • Digital Fountain Approach. • A sender encodes n packets of original content into an unbounded set of encoding packets. • A receiver can reconstruct the original content by receiving any n distinct encoding packets.

10. Overview ROMA C B A 5Mbps 2Mbps

11. Candidate Architectures • Limited Buffer Space solution • Back-pressure mechanism • Unlimited Buffer Space Solution • Use a disk as an extra buffer. • Limitations. • A separate application buffer for each downstream. • Substantial complexity to support I/O access. • The overlay cannot be adaptively reconfigured. Memory Memory Outgoing TCP Memory Incoming TCP

12. Adaptive reconfiguration • Adaptive reconfiguration of overlay network. • Reconfigure when congestion or failures of intermediate nodes occur. Memory A 5Mbps 10Mbps B C X 1Mbps 5Mbps 10Mbps D Shapeshifter BCMR’02

13. Overlay Node Implementation Application Layer Buffer Application Layer Transport Layer Incoming TCP Outgoing TCP

14. Modeling Chains of TCP Connections

15. Chains of TCP Connections • Loosely coupled TCP connections. • An upstream TCP connection may or may not affect the performance of a downstream TCP. • A downstream connection never affects the performance of an upstream connection.

16. Modeling Chains of TCP flows I C B A 5Mbps 2Mbps • When the downstream transfer rate is slower than the upstream transfer rate. • The application layer buffer will grow without bound. • Behaves like a normal TCP driven by an application that always has data to send.

17. Modeling Chains of TCP flows II C B A 2Mbps 5Mbps • When the downstream transfer rate is faster than the upstream transfer rate. • B will periodically drain the application level buffer. • The throughput to C is limited to that of the upstream rate into B.

18. Expected throughput on ROMA RTT_1 RTT_2 RTT_i S 1 2 i P_1 P_2 P_i • Local network condition limits the throughput • of overlay node. • OR the upstream connection limits the • throughput of overlay node.

19. Other Measurement Studies • S. Savage et al. • The end-to-end effects of Internet path selection, ACM SIGCOMM 1999. • There often exists detour route with lower aggregate loss rate and shorter round-trip time than IP rounte. • D. Andersen et al. • Resilient Overlay Networks, SOSP 2001. • Use detour route both to improve performance and to route around faults in the overlay. • Conventional wisdom on overlay network follows this model. Increased latency and loss rate will reduce the performance of overlay node comparing to the direct unicast.

20. Example

21. Experiments • PlanetLab • 160 machines hosted by 65 sites. • Linux. • We use University hosts in the U.S. • Abilene.

22. Multirate Reliable Multicast Effect of Link Stress • Slow link does not impact the performance either at upstream nodes, or at nodes in other regions of the tree

23. Multirate Overlay Multicast Effect of Link Stress

24. Throughput Improvement GT BU UIUC

25. How to construct the overlay tree • Construct the single-source widest-path tree. • Maximize the minimum per-hop available bandwidth to every destination. • Simple variant of Dijkstra’s algorithm. • Weight on each hop is the available bandwidth. • Select the unvisited node with the widest path from the source. • Path width is measured by the minimum of the weights on the path.

26. Maximizing Overall Throughput

27. Throughput Advantage

28. Conclusion • ROMA • New architecture for reliable distribution of large content across an overlay network using TCP. • Multiple-rate reception. • Minimal amount of resources at the application layer. • Provides ability to adaptively reconfigure the topology. • Provides ability to speed up downloads with collaborative peer-to-peer transfers. [BCMR’02] • Analysis of chains of loosely coupled TCP • TCP chains offer an opportunity to increase performance.