Richard Hughes-Jones The University of Manchester hep.man.ac.uk/~rich/ then “Talks”

1. How do transport protocols affect applications&The relative importance of different protocol propertiesPanel Discussion Richard Hughes-Jones The University of Manchesterwww.hep.man.ac.uk/~rich/ then “Talks” PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

2. Panellists • Pascale Primet • INREA, France • Ralph Niederberger • Research Center Juelich, Germany • Tim Sheppard • Katsushi Kobayashi • National Institute Adv. Industrial Science & Technology, Japan • Michael Welzl • University of Innsbruck, Austria PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

3. Some Areas for Discussion • What is the interaction between Application and Transport Protocol? • What is the relative importance of fairness vs throughput? • rtt fairness (OK what is fairness?) • mtu fairness • TCP friendliness • How to AIMD rate fluctuations relate to stability & sharing? • Stability of Achievable Throughput • Does provable stability of protocols matter? • Is the computational complexity of a protocol important? • What is the relative importance of convergence time? • Link utilisation (by this flow or all flows) • Should there be a bias towards "mice“? – Applications • Is conceptual simplicity of the protocol important? PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

4. Action of the transport protocol - help or hindrance to the application ? PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

5. TCP Congestion windowgets re-set on each Request • TCP stack RFC 2581 & RFC 2861 reduction of Cwnd after inactivity • Even after 10s, each response takes 13 rtt or ~260 ms • Transfer achievable throughput120 Mbit/s peak • Event rate very low • Application not happy! Remote Compute Farms: Application Req-Resp • CERN-Manc • Round trip time 20 ms • Web100 hooks for TCP status • 64 byte Request green1 Mbyte Response blue • TCP in slow start • 1st event takes 19 rtt or ~ 380 ms PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

6. VLBI Application Protocol • VLBI signal wave front • Data wave front send to Correlator PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

7. Delay in stream Packet loss Expected arrival time at CBR Visualising CBR/TCP Stephen Kershaw • When packet loss is detected TCP: • Reduces Cwnd • Halves the sending rate • Expect a delay in the message arrival time Arrival time Message number / Time PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

8. CBR/TCP: UKLight JBO-JIVE-Manc • Message size: 1448 Bytes • Wait time: 22 us • Data Rate: 525 Mbit/s • Route:JB-UKLight-JIVE-UKLight-Man • RTT ~27 ms • TCP buffer 32M bytes • BDP @512Mbit 1.8Mbyte • Estimate catch-up possible if loss < 1 in 1.24M Timely data arrival PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

9. And now for the protocols … PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

10. SC2004 Disk-Disk bbftp • bbftp file transfer program uses TCP/IP • UKLight: Path:- London-Chicago-London; PCs:- Supermicro +3Ware RAID0 • MTU 1500 bytes; Socket size 22 Mbytes; rtt 177ms; SACK off • Move a 2 GByte file • Web100 plots: • Standard TCP • Average 825 Mbit/s • (bbcp: 670 Mbit/s) • Scalable TCP • Average 875 Mbit/s • (bbcp: 701 Mbit/s~4.5s of overhead) • Disk-TCP-Disk at 1Gbit/s PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

11. Transport Protocols • TCP • Reno; HS-TCP; Scalable; H-TCP; C-TCP; BIC; CUBIC; LCTP • XCP • UDP • Some applications NEED this form of delivery • RTP / RTSP • Lots of streaming applications available now • DCCP • multicast PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

12. DCCP: Datagram Congestion Control Protocol • Unreliable • No re-transmissions • Has modular congestion control • Can detect congestion and take avoiding action • Different algorithms can be selected – ccid • TCP-like • TCP Friendly Rate Control • DCCP is like UDP with congestion control • DCCP is like TCP without reliability • Application uses • Multi-media – send new data instead of re-sending useless old data • Applications that can choose data encoding & transmission rate • e-VLBI – discussing a special ccid • RFCs 4340, CCIDs RFC 4341 4342 • e-VLBI considering a ccid: UDP with congestion detection – API extension • Detect potential problems with other network users – unexpected route changes PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

13. Fairness and Throughput Smaller RTT is faster ! Larger MTU is faster ! PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

14. Remaining flows do not take up slack when flow removed Increase recovery rate RTT increases when achieves best throughput Congestion has a dramatic effect Recovery is slow Rate fluctuations, Stability & SharingTCP Reno single stream Les Cottrell PFLDnet 2005 • Low performance on fast long distance paths • AIMD (add a=1 pkt to cwnd / RTT, decrease cwnd by factor b=0.5 in congestion) • Net effect: recovers slowly, does not effectively use available bandwidth, so poor throughput • Unequal sharing SLAC to CERN PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

15. Which Protocol for my Network PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

16. Lab to Lab Lightpath • Many application share • Classic congestion points • TCP stream sharing and recovery NEEDED • Advanced TCP stacks Transports for LightPaths • Host to host Lightpath • One Application • No congestion • Lightweight framing PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

17. Transports for Academic Networks • High Bandwidth Backbones • But care needed with Access links – Countries and Campus • Many Application flows • Note the Digital Divide • Roles for Advanced TCP stack and other transports. • Many different technologies – often low Bandwidths • Cautious/conservative Transport Protocols • Standard TCP • Linux & BIC • Microsoft & C-TCP Transports for Global Internet PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

18. Summary: Some Areas for Discussion • What is the interaction between Application and Transport Protocol? • What is the relative importance of fairness vs throughput? • rtt fairness (OK what is fairness?) • mtu fairness • TCP friendliness • How to AIMD rate fluctuations relate to stability & sharing? • Stability of Achievable Throughput • Does provable stability of protocols matter? • Is the computational complexity of a protocol important? • What is the relative importance of convergence time? • Link utilisation (by this flow or all flows) • Should there be a bias towards "mice“? – Applications • Is conceptual simplicity of the protocol important? PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

19. Thanks to the Panellists • Pascale Primet • INREA, France • Ralph Niederberger • Research Center Juelich, Germany • Tim Sheppard • Katsushi Kobayashi • National Institute Adv. Industrial Science & Technology, Japan • Michael Welzl • University of Innsbruck, Austria PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester

20. Delay in stream Packet loss Expected arrival time at CBR CBR/TCP: Catch-up? Stephen Kershaw • If Throughput NOT limited by TCP buffer size / Cwnd maybe we can re-sync with CBR arrival times. • Need to store CBR messages during the Cwind drop in the TCP buffer • Then transmit Faster than the CBR rate to catch up Arrival time Message number / Time PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester