390 likes | 497 Views
A Differentiated Services Implementation for High-Performance TCP Flows. Volker Sander, Ian Foster, Alain Roy and Linda Winkler Forschungszentrum Jülich GmbH Argonne National Laboratory The University of Chicago. Outline. The GARA quality of service architecture
E N D
A Differentiated Services Implementation for High-Performance TCP Flows Volker Sander, Ian Foster, Alain Roy and Linda Winkler Forschungszentrum Jülich GmbH Argonne National Laboratory The University of Chicago
Outline • The GARA quality of service architecture • GARA and network quality of service • Experimental results • Summary and future directions 23-May-2000, TNC 2000
GARA • General-purpose Architecture for Reservation and Allocation • Developed as part of the Globus Project • www.globus.org • Globus Toolkit provides enabling security and directory services • Otherwise independent of Globus • Goal: Provide end-to-end Quality of Service (QoS) to high-end applications 23-May-2000, TNC 2000
Example High-end Applications • Teleimmersion: Virtual reality collaboration • Distributed Scientific Computing • Analysis of large data sets • Real-time simulation steering • Remote control of scientific instruments • Real-time visualization • Collaboration: Multiple people visualizing & steering • We have to address UDP and TCP flows 23-May-2000, TNC 2000
How does GARA Help? • Resource discovery • Network • Computers • Disks • Advance reservations • Security: control over who gets reservations • Monitoring of reservations • Coordination of multiple reservations } Not just Network QoS! 23-May-2000, TNC 2000
Resource Manager • The Resource Manager is the core of GARA • Admission control • Configures resource • Monitors resource • Assumes exclusive access to the resource • Otherwise no guarantees can be made GARA’s network resource manager is a Bandwidth Broker 23-May-2000, TNC 2000
Network Resource Manager • Admission control • Has knowledge of topology within a domain • Tracks reservations on path through domain • Configures routers • Differentiated services (more detail soon) • Currently telnet with command-line interface • COPS/SNMP in the future • Monitors resource • Query edge router for flow statistics • Conforming and exceeding (dropped) packets 23-May-2000, TNC 2000
Network QoS Implementation • We use differentiated services • Expedited forwarding to implement “premium service” • We use Cisco 7507 routers (thanks Cisco!) • Edge routers controlled with resource manager, as described above • Packets that exceed the reservation are dropped 23-May-2000, TNC 2000
The GARNET Testbed 23-May-2000, TNC 2000
Evaluation Tools • UDP Traffic Generator • Modified Version of Andy Adamson’s gen_send and gen_recv • Evaluate admission control • Creating competing traffic • MGEN/Drec • Evaluate Delay and Jitter for Premium UDP Flow • IPERF • Modified Version of ttcp • GARA-enabled (wait for reservation) • Support for a desired application rate • Consecutive bandwidth reporting • Bulk transfer ttcp 23-May-2000, TNC 2000
Basic Experiment I UDP Receiver UDP Sender GARA Diffserv Resource Manager 23-May-2000, TNC 2000
Basic Experiment I • Goal: Proof of Admission Control 23-May-2000, TNC 2000
Basic Experiment II UDP Reveiver UDP Sender UDP Sender UDP-Realtime- Receiver GARA Diffserv Resource Manager 23-May-2000, TNC 2000
Basic Experiment II • Goal: Demonstrate Low-Latency for UDP flows 23-May-2000, TNC 2000
TCP Issue:Exceeding the Reservation 45 Slow Start Exponential Growth 40 35 Congestion Control Linear Growth 30 25 TCP Congestion Window Size 20 15 10 5 0 0 time 23-May-2000, TNC 2000
UDP Sender Basic Experiment III UDP Receiver TCP Receiver GARA Diffserv Resource Manager TCP Sender 23-May-2000, TNC 2000
Conclusions for Implementing a Bandwidth Broker • Avoid any drops if you care about short-term impact • Instead use feedback mechanisms to inform the application / the agent to adapt • its transmission rate • its reservation 23-May-2000, TNC 2000
TCP Issue II • TCP’s Flow Control • Traffic might become bursty if the actual window size is large • Bandwidth*Latency product as minimum window size 23-May-2000, TNC 2000
Demonstration of TCP’s Burstiness 23-May-2000, TNC 2000
Conclusions for Implementing a Bandwidth Broker • Be aware of burstiness • Token bucket depth should allow a full window burst T = Reserved_BW * Estimated_RTT Or implement signaling from the application • How does this interfere with UDP low-latency flows in one aggregate behavior? 23-May-2000, TNC 2000
Basic Experiment IV UDP Receiver UDP Sender UDP-RT Receiver UDP Sender TCP Sender TCP Receiver GARA Diffserv Resource Manager 23-May-2000, TNC 2000
99% WFQ, No Traffic Shaping 23-May-2000, TNC 2000
Conclusions for Implementing a Bandwidth Broker • If Traffic Shaping is not possible, guarantee as much bandwidth to the premium flow as possible. • Admission Control is performed at the edge BUT: Be aware of default queue limits 99% WFQ BW results in a maximum increase of RTT by RTT/2 (assuming 33% EF-Traffic) 23-May-2000, TNC 2000
Standard BE Behavior 23-May-2000, TNC 2000
WFQ: Default Buffer - Just BE 23-May-2000, TNC 2000
WFQ: Modified Buffer - Just BE 23-May-2000, TNC 2000
TCP & Low-Delay Flows • TCP can interfere with UDP flows that want low-delay: • We want traffic shaping to smooth out premium bursts • TCP can be very bursty • Solution: • Traffic Shaping (but…) 23-May-2000, TNC 2000
Traffic Shaping for Entire Premium Class 23-May-2000, TNC 2000
TCP & Low-Delay Flows • Aggregate-based traffic shaping adds delays for low-delay UDP traffic • Solution: • Don’t use a single traffic-shaping configuration for the entire premium class 23-May-2000, TNC 2000
QoS Mechanisms: Inside the Ingress Router 23-May-2000, TNC 2000
Dynamic Traffic Shaping 23-May-2000, TNC 2000
When a reservation ends, the bulk-transfer speeds up When a reservation begins, the bulk-transfer backs off The competitive UDP traffic never interferes Bulk Transfer (LAN) 23-May-2000, TNC 2000
Bulk Transfer (WAN) 23-May-2000, TNC 2000
Then CPU becomes loaded Until competition begins Finally, we turn off network reservation We reserve net We reserve CPU Initially easy to get 80 Mbps Network + CPU Reservations 23-May-2000, TNC 2000
Current Status • A working GARA prototype exists • Differentiated Services • Real-Time CPU Scheduling (DSRT) • DPSS Disk Access • Integrated security, resource discovery, etc. • Many experiments have been performed • Expedited Forwarding is working • Work with early adopters has started • E.g., DOE labs, MREN universities, NASA 23-May-2000, TNC 2000
Future Work • Technology development • More work on Co-reservations • Policy issues: Who has access when; costs, accounting; priorities, preemption • COPS interfaces, inter-domain issues • Experimentation with more real applications • Higher bandwidth flows • MPLS, wavelength allocation 23-May-2000, TNC 2000
Questions? • Feel free to email us: • Volker Sander: v.sander@fz-juelich.de • Ian Foster: foster@mcs.anl.gov • Alain Roy: roy@mcs.anl.gov • Linda Winkler: winkler@mcs.anl.gov • Check our web site: • http://www.mcs.anl.gov/qos/ • Numerous technical papers available 23-May-2000, TNC 2000