150 likes | 164 Views
UDTv4 is a protocol designed for fast, reliable, and secure data transfer. It features a three-layer design, improved resource management, and faster connection setup time. This version also includes better security features. Evaluation and future plans are discussed.
E N D
UDTv4: Improvements on Performance and Scalability Yunhong Gu and Robert Grossman National Center for Data Mining Univ. of Illinois at Chicago
Overview • Previous Versions • Use Case (Sector & Sphere) • Protocol Design & Implementation • Evaluation
UDT: A Brief History • Started in 2001 (SABUL) • Bulk data transfer: control information over TCP, data over UDP • Naïve congestion control • UDTv2 • Both control and data over UDP • Fast, fair, and friendly congestion control • UDTv3 • Configurable congestion control • Partial reliability & messaging
Use Case: Sector & Sphere • http://sector.sf.net • Distributed storage/file system • Distributed computing framework (data processing)
Sector & Sphere Cloud Computing • http://www.opencloudconsortium.org • 120-node over wide area (Baltimore, Chicago, and San Diego) • 10Gb/s inter-site bw, 2Gb/s inter-node bw • Data access: user may upload and download data • Data exchange: data or computing result can be moved between any pair of nodes
Sector & Sphere Cloud Computing • Large number of concurrent connections • Connections join and leave frequently and randomly • Data transfer happens in complicated patterns
UDTv4: What’s New? • Three-layer design • UDP multiplexer • UDT flow • UDT connection • Support more concurrent UDT connections • Resource management • Sharing congestion control instance • Faster connection setup time • Better security
Connection Record Index/Cache • Store historical and statistical information about each peer address • RTT, bw, loss rate, etc. • Garbage collection
Implementation • Scalability and resource management • Queue management for UDT flows • Sending queue: O(logN), N is the number of flows • Receiving queue: O(1) • Buffer sharing and auto-sizing • Security • Rendezvous connection setup • Defend buffer overflow, SYN flooding, malicious control or data packets
Current Status and Future Plan • Current version 4.2 • Version 4.3 to be released before SC08 • Congestion control sharing not available in stable release yet • Internet Draft (version 2), will try IETF standardization after v5
For More Information • Web: http://udt.sf.net Open source project (C++) • Email: Yunhong Gu gu@lac.uic.edu