1 / 45

UDT: UDP based Data Transfer

UDT: UDP based Data Transfer. Yunhong Gu & Robert Grossman Laboratory for Advanced Computing University of Illinois at Chicago.

ulric
Download Presentation

UDT: UDP based Data Transfer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. UDT: UDP based Data Transfer Yunhong Gu & Robert Grossman Laboratory for Advanced Computing University of Illinois at Chicago The work is supported by the 2/032/2004 ELTE-BUTE-Ericsson NKFP project on  Research and Developments of Tools Supporting Optimal Usage of Heterogen  Communication Networks Németh Felicián, Tarján Péter

  2. Outline • Background • UDT Protocol • UDT Congestion Control • Implementation/Simulation Results • Summary PFLDnet 2004

  3. Background • Distributed data intensive applications over wide area optical networks: • Grid computing, access of bulk scientific data, data mining, high resolution video, etc. • Transport protocol support: • Efficient and fair bandwidth unitization • TCP does not work! PFLDnet 2004

  4. Requirements to the New Protocol • FAST • High utilization of the abundant bandwidth either with single or multiplexed connections • FAIR • Intra-protocol fairness, independent of RTT • FRIENDLY • TCP compatibility PFLDnet 2004

  5. Use Scenarios • Small number of sources shares abundant bandwidth • Bulk data transfer • Most of the packets can be packed in maximum segment size (MSS) in a UDT session • MSS can be set up by applications and the optimal value is the path MTU PFLDnet 2004

  6. UDT History • 2000: SABUL Concept • 2001: SABUL version 1.0 • 2002: dSABUL • 2002: SABUL version 2.0, 2.1, 2.2, 2.3, • 2003: UDT 1.0 • 2004: UDT 1.1 & 1.2 • 2004: UDT 2.0 • 2005: UDT 3.0 PFLDnet 2004

  7. Papers • Supporting Configurable Congestion Control in Data Transport ServicesYunhong Gu and Robert L. GrossmanUIC/LAC Technical Report, in submission • Experiences in Design and Implementation of a High Performance Transport ProtocolYunhong Gu, Xinwei Hong, and Robert L. GrossmanSC 2004, Nov 6 - 12, Pittsburgh, PA, USA. • An Analysis of AIMD Algorithms with Decreasing IncreasesYunhong Gu, Xinwei Hong and Robert L. GrossmanFirst Workshop on Networks for Grid Applications (Gridnets 2004), Oct. 29, San Jose, CA, USA. • Optimizing UDP-based Protocol ImplementationYunhong Gu and Robert L. GrossmanPFLDNet 2005, Lyon, France, Feb. 2005 • SABUL: A Transport Protocol for Grid ComputingYunhong Gu and Robert L. Grossman, Journal of Grid Computing, 2003, Volume 1, Issue 4, pp. 377-386 PFLDnet 2004

  8. What’s UDT? • UDT: UDP based Data Transfer • Reliable, application level, duplex, transport protocol, over UDP with congestion control • Implementation: Open source C++ library • Two orthogonal parts • The UDT protocol framework that can be implemented above UDP, with any suitable congestion control algorithms • The UDT congestion control algorithm, which can be implemented in any transport protocols such as TCP PFLDnet 2004

  9. Congestion Control Schemes • Window control sends data in bursts • TCP pacing (decreases throughput) • Rate control • can lead to continuous loss • Rate control + Supportive window control PFLDnet 2004

  10. Packet Structure • Data Packet: • Header: 1bit flag + 31bit sequence number • Control Packet: • Header: 1bit flag + 3bit type + 12bit reserved + 16bit ACK seq. no. + (0 - 32n)bit control info • Type: ACK, ACK2, NAK, Handshake, Keep-alive, and Shutdown • Actual size of a UDT packet can be ascertained from UDP header PFLDnet 2004

  11. Data Packet • Flag Bit: 0 • UDT uses 31-bit packet based sequence number, ranging from 0 and (231 - 1) • Sequence number may be wrapped if it exceeds the maximum available number PFLDnet 2004

  12. Control Packet • Flag Bit: 1 • type: 3-bit • handshake (000), shutdown (101), keep-alive (001) • ACK (010), ACK2 (110), NAK (011) • UDT uses sub-sequencing: each ACK and related ACK2 are assigned a 16-bit unique ACK sequence number PFLDnet 2004

  13. Acknowledgements • Selective acknowledgement (ACK) • Generated at every constant interval to send back largest continuously received sequence number of data packets. • The sender sends back an ACK2 to the receiver for each ACK (sub-sequencing). • Also carries RTT, packet arrival speed, and estimated link capacity. • Explicit negative acknowledgement (NAK) • Generated as soon as loss is detected. • Loss information may be resent if receiver has not received the retransmission after an increasing interval. • Loss information is compressed in NAK. PFLDnet 2004

  14. Timing • Packet Scheduling Timer • Tuned by Rate Control • High precision in CPU clock cycles • Implementation depends on self-clocking,packet sending burst • Rate Control Timer: trigger rate control • RCTP = 0.01 seconds • ACK Timer: trigger acknowledgement • ATP = RCTP PFLDnet 2004

  15. Timing (cont.) • NAK Timer: trigger negative acknowledgement • NTP = RTT • Retransmission Timer: trigger retransmission based on time-out and maintain connection status • RTP = (exp-count + 1) * RTT + ATP where exp-count is the number of continuous time-out PFLDnet 2004

  16. Pkt. Scheduling Timer Sender Sender Sender DATA Recver Recver ACK ACK2 ACK Timer NAK NAK Timer Retransmission Timer Rate Control Timer UDT Architecture PFLDnet 2004

  17. Congestion Control • Rate based congestion control (Rate Control) • RC tunes the packet sending period. • RC is triggered periodically at the sender side. • RC period is constant of 0.01 seconds. • Window based flow control (Flow Control) • FC limits the number of unacknowledged packets. • FC is triggered on each received ACK at the sender side. PFLDnet 2004

  18. … … Rate Control • AIMD: Increase parameter is related to link capacity and current sending rate; Decrease factor is 1/9, but not decrease for all loss events. • Link capacity is probed by packet pair, which is sampled UDT data packets. • Every 16th data packet and it successor packet are sent back to back to form a packet pair. • The receiver uses a median filter on the interval between the arrival times of each packet pair to estimate link capacity. PFLDnet 2004

  19. Receiver Based Packet Pair (RBPP) • OK in high-speed optical links • Not working well in certain cases • multi-channel links • Underestimation • UDT become conservative • Overestimation • UDT become more aggressive. • Finally, UDT turns into window-based cong. control • V. Paxson, End-to-End Internet Packet Dynamics, IEEE/ACMTransactions on Networking, Vol.7, No.3, pp. 277-292, June 1999. PFLDnet 2004

  20. Rate Control (cont.) 1. If loss rate is greater than 1%, do not increase; 2. Number of packets to be increased in next RCTP time is: where B is estimated link capacity, C is current sending rate. Both are in packets or packets per second. MSS is the packet size in bytes. β = 1.5 * 10-6. 3. Recalculate packet sending period (STP). An Analysis of AIMD Algorithms with Decreasing Increases Yunhong Gu, Xinwei Hong and Robert L. Grossman First Workshop on Networks for Grid Applications (Gridnets 2004), Oct. 29, San Jose, CA, USA. PFLDnet 2004

  21. Rate Control (cont.) B = 10Gbps, MSS = 1500 bytes PFLDnet 2004

  22. UDT Algorithm PFLDnet 2004

  23. Rate Control (cont.) • Decrease sending rate by 1/9, (or equivalently, increase packet sending period by 1.125), only if • Received an NAK, whose last lost sequence number is greater than the largest sequence number when last decrease occurred; or • The number of loss events since last decrease has exceeded a threshold, which increases exponentially and is reset when condition 1 is satisfied. • No data will be sent out for the next RCTP time if a decrease occurs. • Help to clear congestion. • In a short period, loss rate due to congestion is larger than loss rate due to physical link error PFLDnet 2004

  24. Flow Control BDP • W = W*0.875 + AS*(RTT+ATP)*0.125 • AS is the packets arrival speed at receiver side. • The receiver records the packet arrival intervals. AS is calculated from the average of latest 16 intervals after a median filter. • It is carried back within ACK. PFLDnet 2004

  25. Slow Start • Flow window starts at 2 and increases to the number of acknowledged packets, until the sender receives an NAK or reaches the maximum window size, when slow start ends. • Packet sending period is 0 during slow start phase and set to the packet arrival interval at the end of the phase. • Slow start only occurs at the beginning of a UDT session. PFLDnet 2004

  26. Implementation: Performance PFLDnet 2004

  27. Implementation: Intra-protocol Fairness PFLDnet 2004

  28. Implementation: TCP Friendliness PFLDnet 2004

  29. Implementation: TCP Friendliness (cont.) PFLDnet 2004

  30. Implementation: File Transfer 1Gbps/15.9ms 1Gbps/110ms Canarie StarLight SARA Disk R: 800Mbps W: 550Mbps Disk R: 800Mbps W: 500Mbps Disk R: 1300Mbps W: 900Mbps PFLDnet 2004

  31. Simulation: UDT Throughput at Different Bandwidth and RTT PFLDnet 2004

  32. Simulation: Performance of Concurrent UDT Flows PFLDnet 2004

  33. Simulation: Intra-protocol Fairness PFLDnet 2004

  34. Simulation: RTT Independence PFLDnet 2004

  35. Simulation: TCP Friendliness PFLDnet 2004

  36. Simulation: Convergence/Stability PFLDnet 2004

  37. 100 100 100 10 50      Simulation: Complex Scenario Link capacity Mbps Node DropTail Flow and its ID PFLDnet 2004

  38. B 200 A 200 x C Simulation: Multi-bottleneck PFLDnet 2004

  39. Summary • UDT Protocol • Application level upon UDP • Selective acknowledgement / explicit negative acknowledgement • UDT Congestion Control • Rate Control • Bandwidth estimation for fast probing available bandwidth and fast recovery • AIMD for fairness • Constant rate control interval • Flow Control • Dynamic flow window according to packet receiving speed PFLDnet 2004

  40. UDT Characters • Good use of available bandwidth • Application level - no changes in router and operating system • No manual tuning • Fair and Friendly: intra-protocol fairness, TCP friendliness, and RTT independence. • Open source PFLDnet 2004

  41. Thank You! LAC: www.lac.uic.edu UDT: sourceforge.net/projects/dataspace Internet Draft: draft-gg-udt-01.txt

  42. Q&A

  43. Comparison: mostly “Related Works” x: packet sending rate PFLDnet 2004

  44. SYS interval: 10 ms • Acceptable trade-off between efficiency and fairness • Intra-protocol and TCP • Larger values • Less responsive to network change • Slower in loss recovery • More stable • Friendlier to TCP • Smaller: the opposite PFLDnet 2004

  45. Target environment • Large BDP networks, • Bulk data transfer • Speed of the control flow: ~4.5 kBps • Who use it? • Using Teraflows to Transport Sloan Digital Sky Survey (SDSS) Data • http://sdss.ncdm.uic.edu:8080/sdss/ • e-VLBI • compares Tsunami, UDT, FAST TCP • http://apan.net/meetings/honolulu2004/materials/engineering/8-APAN2.ppt PFLDnet 2004

More Related