230 likes | 313 Views
Buffer requirements for TCP: queueing theory & synchronization analysis. Gaurav Raina Damon Wischik Cambridge UCL. TCP. if (seqno > _last_acked) { if (!_in_fast_recovery) { _last_acked = seqno; _dupacks = 0; inflate_window(); send_packets(now); _last_sent_time = now; return;
E N D
Buffer requirements for TCP:queueing theory & synchronization analysis Gaurav Raina Damon WischikCambridge UCL
TCP if (seqno > _last_acked) { if (!_in_fast_recovery) { _last_acked = seqno; _dupacks = 0; inflate_window(); send_packets(now); _last_sent_time = now; return; } if (seqno < _recover) { uint32_t new_data = seqno - _last_acked; _last_acked = seqno; if (new_data < _cwnd) _cwnd -= new_data; else _cwnd=0; _cwnd += _mss; retransmit_packet(now); send_packets(now); return; } uint32_t flightsize = _highest_sent - seqno; _cwnd = min(_ssthresh, flightsize + _mss); _last_acked = seqno; _dupacks = 0; _in_fast_recovery = false; send_packets(now); return; } if (_in_fast_recovery) { _cwnd += _mss; send_packets(now); return; } _dupacks++; if (_dupacks!=3) { send_packets(now); return; } _ssthresh = max(_cwnd/2, (uint32_t)(2 * _mss)); retransmit_packet(now); _cwnd = _ssthresh + 3 * _mss; _in_fast_recovery = true; _recover = _highest_sent; } traffic rate [0-100 kB/sec] time [0-8 sec]
Motivation: buffer size • Internet routers have buffers, to accomodate bursts in traffic. • How big do the buffers need to be? • 3 GByte? Rule of thumb is C*RTT—what Cisco does today • 300 MByte? C*RTT/√N [Appenzeller, Keslassy, McKeown, 2004 ] • 30 kByte? constant buffer size, independent of line rate • Large buffers are unsustainable: • Data volumes double every 10 months • CPU speeds double every 18 months • Memory access speeds double every 10 years
Desynchronized TCP flows: aggregate traffic is smooth small buffers are OK SynchronizedTCP flows: aggregate traffic is bursty large buffers are needed + + = Buffers & synchronization + individualflow rates + = aggregatetraffic rate time
Network traffic model • When there are many TCP flows, the average traffic rate xt varies smoothly, according to a differential equation[Misra, Gong, Towsley, 2000] • involving average round trip time RTT • and packet loss probability pt aggregatetraffic rate desynchronized synchronized time
Network traffic model • When there are many TCP flows, the average traffic rate xt varies smoothly, according to a differential equation[Misra, Gong, Towsley, 2000] • involving average round trip time RTT • and packet loss probability pt • The packet loss probability pt satisfies a differential equation, which depends on buffer size:[Raina, W., 2005] • either for small buffers (max queueing delay «RTT) • or for large buffers (max queueing delay RTT) queuesize time
Analysis • Write down differential equations • for average TCP traffic rate xt • for queue dynamics and loss prob pttaking account of buffer size • Calculate • average link utilization • average queue occupancy/delay • extent of synchronizationand consequent loss of utilization, and jitter
Results There is a virtuous circle:desynchronization permits small buffers; small buffers induce desynchronization.
Illustration: 20 flows Standard TCP, single bottleneck link, no AQMservice C=1.2 kpkt/sec, RTT=200 ms, #flows N=20 B=20 pkts(small buffer) B=54 pkts(intermediate buffer) B=240 pkts(large buffer)
Illustration: 200 flows Standard TCP, single bottleneck link, no AQMservice C=12 kpkt/sec, RTT=200 ms, #flows N=200 B=20 pkts(small buffer) B=170 pkts(intermediate buffer) B=2,400 pkts(large buffer)
Illustration: 2000 flows Standard TCP, single bottleneck link, no AQMservice C=120 kpkt/sec, RTT=200 ms, #flows N=2000 B=20 pkts(small buffer) B=537 pkts(intermediate buffer) B=24,000 pkts(large buffer)
Limitations/concerns • Surely bottlenecks are at the access network, not the core network? • Unwise to rely on this! • The small-buffer theory works fine for as few as 20 flows • Perhaps TCP is constrained by receiver window, not by congestion window? • Unwise to rely on this! Probably there are congestion-constrained flows which cause congestion • Would small buffers hurt flows that are constrained by receiver window?Not if they respond appropriately.
Further work • These results have been partially validated (Internet2 & Level3) • Full validation needs • goodly amount of traffic • full measurement kit • ability to control buffer size • Alternatives to TCP • should pace packet injections • can achieve desynchronization at all window sizes
Small buffers As line rate increases • queue size fluctuates more and more rapidly • so pt depends only on the instantaneous utilization at time t • can model packet arrivals by a Poisson process of rate xt[Cao+Ramanan 2002] • queue size distribution depends only on link utilization,not on line rate queueing delay19 ms queueing delay1.9 ms queueing delay0.19 ms queue size[0-15 pkt] time [0-5 sec]
Large buffers (queueing delay 200 ms) • When Nxt<Cthe queue size is small (C=line rate) • No packet drops, so TCP increases xt queue size[0-160 pkt] time [0-10 sec]
Large buffers (queueing delay 200 ms) • When Nxt<Cthe queue size is small (C=line rate) • No packet drops, so TCP increases xt • When Nxt>C the queue fills up and packets begin to get dropped queue size[0-160 pkt] time [0-10 sec]
Large buffers (queueing delay 200 ms) • When Nxt<Cthe queue size is small (C=line rate) • No packet drops, so TCP increases xt • When Nxt>C the queue fills up and packets begin to get dropped • TCP may ‘overshoot’, leading to synchronization queue size[0-160 pkt] time [0-10 sec]
Large buffers (queueing delay 200 ms) • Drop probability depends onboth average traffic rate xt and queue size qt queue size[0-160 pkt] time [0-10 sec]
System summary • xt = average traffic rate at time tpt = packet loss probability at time tC = line rate B = buffer size RTT = round trip time N=# flows • TCP traffic model • Small buffer queueing model(though this is sensitive to traffic statistics) • Large buffer queueing model
Stability/instability analysis • For some values of RTT*C/N, the dynamical system is stable • For others it is unstable and there are oscillations(i.e. the flows are partially synchronized) • When it is unstable, we can calculate the amplitude of the oscillations trafficrateNxt/C time
Instability plot traffic intensity Nx/C extent ofoscillationsin Nx/C TCP throughput equation log10 ofpkt lossprobability p queue equation
Instability plot traffic intensity Nx/C RTT*C/N=4pkts log10 ofpkt lossprobability p RTT*C/N=20 pkts RTT*C/N=100 pkts
Alternative buffer-sizing rules Intermediate buffers buffer = bandwidth*delay / sqrt(#flows)orLarge buffers, no AQM buffer = bandwidth*delay Large buffers with RED buffer=bandwidth*delay*{¼,1,4} Small buffers, no AQM buffer={10,20,50} pkts Small buffers, no AQM, ScalableTCP buffer={50,1000} pkts