TCP Westwood: Experiments over Large Pipes

1. TCP Westwood: Experiments over Large Pipes Cesar Marcondes Anders Persson Prof. M.Y. Sanadidi Prof. Mario Gerla NRL – Network Research Lab UCLA

2. Background • TCP NewReno is challenged on large pipes: • Slow convergence to full utilization • Not intended to handle non-congestion packet loss • Large Pipes performance criteria: • Utilization • Stability • Fast Ramp Up to “Cruising Speed” from Slow start • Fairness under differing RTTs • Friendliness to NewReno • Alternatives include: HS TCP, FAST, TCPW • Goal of this study: Measurements of TCPW, FAST and HS TCP over large pipes PATHNETS 2004 - San Jose CA

3. TCPW • Goal: high utilization, fairness, and friendliness over large leaky dynamic pipes • Sender side only estimation of Eligible Rate Estimate (ERE) • Estimation takes into account congestion level, capacity of the bottleneck, achieved rate • Exponential filtering to time average estimates and avoid network conditions instability • ERE is used to: • (1) set congestion window after packet loss • (2) repeatedly reset ssthresh to reach “cruising speed” fast from slow start PATHNETS 2004 - San Jose CA

4. Tk Tk TCPW ABSE RE Sampling: Packet train, fair estimate under congestion, underestimates under random loss BE Sampling: Packet pair, effective under random loss, overestimates under congestion Under No Congestion Under Congestion • To obtain ERE: adapt the sample interval Tk according to congestion level • Congestion level is similar to that in Vegas: Expected Rate-Achieved Rate PATHNETS 2004 - San Jose CA

5. Experiments Environment (Powerful Machines) CPU: Xeon 3.06GHz Cache: 512 L2/ 1MB L3 Intel 1000PRO PCI-X BUS 133MHz NewReno Sender Gigabit link Internet2 NewReno Receiver (Alabama) Gigabit link UCLA Gigabit Switch Advanced TCP Sender NewReno Receiver (Caltech) PATHNETS 2004 - San Jose CA

6. UCLA Internet2 Link Traffic Other UCLA Users in Background Our Experiments Traffic PATHNETS 2004 - San Jose CA

7. Test Methodology • Automated Scripts • Scheduled by Unix crontab • Automatically reinitiate the O.S. with each protocol and conduct new measurements • Linux: FAST, HS-TCP and NewReno • FreeBSD: TCPW • Sender/Receiver buffer is set to 2 MB to enable high utilization of Gbps links • Iperf traffic generation, TCPdump, Nistnet emulator PATHNETS 2004 - San Jose CA

8. Benchmark Tests • Case Study I: • UCLA-Alabama (155 Mbps, 64 msec) • Case Study II: • UCLA-CalTech (1 Gbps, 4msec) • Group of 10 successive night time runs for each test • Throughput, fairness, friendliness • Artificial non-congestion loss (PER 0.1 to 0.5%) PATHNETS 2004 - San Jose CA

9. Case Study I: UCLA–Alabama NewReno Sender Internet2 (Gigabit) ATM Atlanta – Alabama NewReno Receiver (Alabama) Advanced TCP Sender 155Mbps ATM Link Bottleneck Link as measured by PathRate And confirmed later by the network admin PATHNETS 2004 - San Jose CA

10. UCLA-Alabama Throughput • Convergence to cruising speed varies among protocols • High deviation among multiple runs in HSTCP and NewReno • HSTCP deviations decrease over time (as the AIMD behavior changes) PATHNETS 2004 - San Jose CA

11. UCLA-Alabama PATHNETS 2004 - San Jose CA

12. UCLA-Alabama Transfer Completion Times • On average: • TCPW and FAST: 0 to 100 MB in 5.8 Sec! • HSTCP: 0 to 100 MB in 7.5 Sec! • NewReno: 0 to 100 MB in 11 Sec! PATHNETS 2004 - San Jose CA

13. UCLA-Alabama Friendliness PATHNETS 2004 - San Jose CA

14. UCLA-Alabama TCP FAST – Preliminary Analysis Outstanding Window as Observed by TCPdump RTT Variation over Time as Observed by TCPdump PATHNETS 2004 - San Jose CA

15. UCLA-Alabama Random Loss Emulation • Induced non-congestion packet loss in emulator (PER 0.1% up to 0.5%) • TCPW throughput much higher than all other schemes UCLA – Alabama NewReno Receiver (Alabama) Advanced TCP Sender Nistnet Network Emulator PATHNETS 2004 - San Jose CA

16. UCLA-Alabama Random Loss Emulation (Results) PATHNETS 2004 - San Jose CA

17. Case Study II: UCLA–CalTech NewReno Sender (UCLA) Internet2 (Gigabit) Advanced TCP Sender (UCLA) TCP Receiver (CalTech) 1 Gbps 4 ms PATHNETS 2004 - San Jose CA

18. UCLA-CalTech Throughput • TCP NewReno starts-up really high since it relies in the cached threshold and the feedback is really fast • Cached Slow Start Threshold versus Adaptive Start-Up (Pros and Cons) • Westwood is delayed by its own Stability Filter • Stability-based Filter dampens estimates in proportion to the variance of observation PATHNETS 2004 - San Jose CA

19. UCLA-CalTech PATHNETS 2004 - San Jose CA

20. UCLA-CalTech TCP Westwood Stability Filter versus Fixed Gain Filter • Sample Estimations vary a lot due to NIC coalescing and OS issues at Gigabit/s. • As variability increases, stability filter relies on a more *stable* moving average filter • Solution: Use a fixed gain instead of an adaptive when we know we are dealing with Gbps range speeds • TCPW ramp up as HS-TCP and FAST PATHNETS 2004 - San Jose CA

21. UCLA-CalTech TCPW Start-Up using Fixed Exponential Average PATHNETS 2004 - San Jose CA

22. UCLA-CalTech Friendliness PATHNETS 2004 - San Jose CA

23. TCPW and FAST performed equally well in terms of average throughput All Advanced TCP protocols have an excellent intra-protocol fairness Friendliness FAST appears to suffer a synchronization problem Under non-congestion error scenario, TCPW shows greater robustness At Gigabit speed, measurements could be messed up by Interrupt Coalescing and other HW/Kernel bottlenecks, affecting moving average filters Conclusions PATHNETS 2004 - San Jose CA

24. Future Work • New algorithm that is Interrupt Coalescence-Aware for Gbps environment • New Agile and Stable Filter • Improve the Automated TCP Test Tool (Benchmark and New Tests) PATHNETS 2004 - San Jose CA

25. Thanks • Netlab CalTech • Xiaoyan Hong – CS / Alabama Univ. PATHNETS 2004 - San Jose CA