Why is the Internet traffic bursty in short time scales?

1. Why is the Internet traffic bursty in short time scales? Constantine Dovrolis Hao Jiang College of Computing Georgia Institute of Technology

2. The many faces of traffic burstiness • FACT: Internet traffic is bursty in very wide range of time scales (microseconds to hours) • “Burstiness” can be related to different statistical aspects of the traffic process: • Variance of marginal distribution in certain time scale • E.g., Gamma renewal process is burstier than Poisson process (with same average rate) • Strong correlations in interarrivals or counts • E.g., packet-trains or ON-OFF user behavior • Scaling behavior in a range of time scales • E.g., IID process: variance decreases with time scale T-1 • E.g., Self-similar process: variance decreases with T-2(1-H), H: Hurst parameter (0.5<H<1)

3. Burstiness in large/medium time scales • Well understood cause-effect relationship in large scales (> seconds) • Leland et al. (TNet’94): asymptotically self-similar scaling and LRD behavior, assuming stationarity • Major causes: • Willinger et al. (Sigcomm’95): heavy-tailed ON-OFF behavior in packet interarrivals • Crovella/Bestavros (TNet’99): heavy-tailed ON-OFF behavior in Web transfers • Rather well understood behavior also in medium scales (in the order of seconds): • Figueiredo et al. (CompNets’02): pseudo-self similar behavior in a range of medium time scales • From a single RTT to tens/hundreds of RTTs • Cause: strong correlations due to TCP congestion control and timeouts

4. Burstiness in short (sub-RTT) time scales • Several (rather contradictory) proposed traffic models: • Multifractal & non-Gaussian model (Riedi et al. TransIT‘99, Feldmann et al. Sigcomm’98.) • Monofractal & Gaussian (Zhang et al. Infocom’03) • Cluster process with Gamma interarrivals (Hohn et al.TransSP’03) • Nonstationary Poisson (Karagiannis et al. Infocom’04) • More importantly, however, the open question is: What causes short-scale (sub-RTT) burstiness? • TCP ACK compression (Feldmann et al. Sigcomm ’99) • “Dense” (i.e., highly variable) interarrivals (Zhang et al. Infocom ’02) • Our objectives: • Identify major cause(s) for sub-RTT burstiness of aggregate Internet traffic • Relate flow characteristics (capacity, RTT, window size, flow size) with resulting burstiness • Explain why some of the previous measurements resulted in contradictory models

5. Main questions • What is the major cause for sub-RTT burstiness in aggregate Internet traffic? • Individual flows or of aggregate traffic? • If individual TCP flows cause sub-RTT burstiness, which component of the TCP protocol is to blame? • Congestion control? Self-clocking? ACK compression? • Which TCP flows are mostly responsible for short scale burstiness? • Large? Dense? High-capacity? • Is there a practical way to reduce short scale burstiness? • Traffic shaping? Change in TCP? • What is the effect of traffic multiplexing (aggregation) on short scale burstiness? • Does aggregation produce “Poisson-like” traffic? • What is the impact of short scale burstiness on queueing performance? • Conventional wisdom: “LRD behavior dominates queueing performance”

6. Overview • Wavelet-based MRA and burstiness definition • Sub-RTT ON-OFF behavior due to TCP self-clocking • Case study: the burstiness of an OC-48 trace • Smoothing effect of TCP pacing

7. Multi-Resolution Analysis (MRA) Analyze variability of traffic process in successive time scales Tj = 2j T0:

8. Wavelet-based MRA (Abry-Veitch) Traffic process in time scale Tj = 2j T0: where Xjkis amount of traffic in Energy of Xj(for Haar wavelet): Energy of Poisson process of rate λ: Ej = λT0 (constant) Energy of periodic process in scale Tj: Ej = 0 (a periodicity causes energy drop) Energy plot: logEj vs j, j=0,1,2,…

9. An unusual definition of traffic burstiness • A traffic process Xj is bursty at scale j if the energy of Xj is higher than the energy of Poisson process with same average rate • Example: energy plot of an OC-48 Abilene trace

10. Overview • Wavelet-based MRA and a burstiness definition • Sub-RTT ON-OFF behavior due to TCP self-clocking • Case study: the burstiness of an OC-48 trace • Smoothing effect of TCP pacing

11. TCP self-clocking • Can a single TCP flow create bursty traffic in sub-RTT scales, and if so, under which condition? • Main TCP parameters: • W: send-window size (segs) T: round-trip time • L: MSS C: path capacity • CT: bandwidth-delay product • Ideal TCP sending behavior within a single RTT: • Driven by self-clocking and delayed-ACKs • Basic operation: each (new) received ACK triggers the back-to-back transmission of at least two new segments • Interarrivals of back-to-back segments after bottleneck: L/C • Interarrivals of generated ACKs at receiver: 2*L/C • Without ACK compression, ACKs will arrive at sender with same interarrival

12. Almost all interarrivals of data segments are equal to L/C • The only interarrivals that are larger than L/C are between successive windows

13. Self-clocking model without cross traffic: case 1 WL < CT, one level ON/OFF

14. Self-clocking model without cross traffic: case 2 WL >= CT, periodic interarrivals

15. Vantage point: about 50% of interarrivals at L/Cp due to delayed ACKs • Window is split in clusters of bursts, due to interfering cross traffic

16. Self-clocking with cross traffic • Within each RTT, send-window is segmented into a number of bursts, with each burst being two or more packets sent back-to-back • Basically, a two-level ON-OFF process • Window duration: Δ , K bursts, burst length: Bi

17. Energy plot for TCP self-clocking with cross traffic

18. Summary so far.. • If TCP’s send-window is less than the flow’s bandwidth-delay product (W < CT), TCP generates bursty traffic in sub-RTT scales • Without cross traffic, one-level ON-OFF process • With cross traffic, two-level ON-OFF process • Otherwise, if W > CT, traffic is almost periodic • Can we verify the previous observations based on the analysis of a real Internet trace?

19. Overview • Wavelet-based MRA and a burstiness definition • Sub-RTT ON-OFF behavior due to TCP self-clocking • Case study: the burstiness of an OC-48 trace • Smoothing effect of TCP pacing

20. Case study: burstiness of OC-48 trace • Goal: identify minimal set of flows that are responsible for short scale burstiness in aggregate traffic • Also, relate flow characteristics (RTT, capacity, flow size, window size) with the resulting energy plot of the aggregate traffic • Definitions: • Mice: flow size < 15KB • Burstiness ratio = CT/W

21. Traffic is bursty across (almost) all time scales • Energy plots of original traffic and TCP subset are the same • Non-TCP traffic does not affect burstiness of aggregate

22. Distribution of per-flow RTTs • Estimation technique: Jiang-Dovrolis, ACM CCR’02

23. Distribution of per-flow capacities • Estimation technique: see Jiang-Dovrolis PAM’04

24. The dip in the 12-th scale reflects a periodicity around 200ms • Weighted average RTT of TCP traffic

25. Large flows determine the shape of the aggregate energy plot • Small flows are also bursty, but they do not affect the energy plot of the aggregate traffic

26. Distribution of BDP/W ratio for large flows • Ratio is quite larger than 1.0 for most of the large flows • As little as 5% traffic has ratio close to 1.0

27. Bulk flows with large BDP/W ratio create sub-RTT burstiness • Bulk flows with small BDP/W ratio are smooth in sub-RTT scales

28. Summary so far.. • Analysis of real aggregate Internet traffic confirms that short scale burstiness is due to the following kind of traffic: • Individual TCP flows • Large size (bulk transfers) • Large BDP relative to the average window size • Also, the extent of the short scale burstiness is related to the (effective) RTT of the TCP traffic • Identified in the energy plot as a dip at that time scale

29. Overview • Wavelet-based MRA and a burstiness definition • Sub-RTT ON-OFF behavior due to TCP self-clocking • Case study: the burstiness of an OC-48 trace • Smoothing effect of TCP pacing

30. Smoothing effect of TCP pacing • Fundamental issue with TCP self-clocking: • May send a window as a burst or a cluster of bursts • Packet transmissions are not “spread” during RTT • Pacing is an alternative to self-clocking: • Transmit packets periodically during RTT • Driven by OS timer at sender • Ideal pacing • Arbitrarily small granularity • But timer overhead is too high • Practical pacing • Timer granularity Tc is typically 1ms or 10ms • Send m packets every nTc time units

31. Ideal pacing of individual TCP flows makes traffic smoother than Poisson in sub-RTT scales

32. The 1ms timer is effective in reducing short scale burstiness • The 10ms timer may be unable to eliminate short scale burstiness

33. Conclusions • What is the major cause for short scale burstiness in aggregate Internet traffic? • ON-OFF packet transmission pattern in large TCP flows • If individual TCP flows cause most of the burstiness, which component of the TCP protocol is to blame? • Self-clocking • Which TCP flows are mostly responsible for short scale burstiness? • Large TCP flows with bandwidth-delay product > average flow’s window • Is there a practical way to reduce short scale burstiness? • Yes, pacing at the sender instead of self-clocking • What is the effect of traffic multiplexing (aggregation) on short scale burstiness? • Aggregation reduces the CoV of the marginal distribution, but it does not remove the correlations in the packet interarrivals (i.e., the traffic does not converge to Poisson process) • What is the impact of short scale burstiness on queueing performance? • Sub-RTT burstiness is important in moderate load conditions, but also in high-load conditions when the bottleneck buffer is small

34. Thanks

35. more slides

36. Background and related work • Wavelet-based multi-resolution analysis • Sub-RTT ON/OFF behavior due to TCP self-clocking • Effects of aggregation • Case study: the burstiness of an OC-48 trace • Smoothing effect of TCP pacing • Queueing performance

37. Effect of aggregation Aggregate if X and Y independent • Flows that do not have significant energy relative to the aggregate do not have a major impact on the burstiness of the aggregate

38. Energy plot of aggregate maintains the shape as that of a single constituent, independent of N • Correlation does not die out with degree of aggregation although statistical multiplexing gain exists

39. Point process theorem: the aggregation of N independent point processes converges to Poisson process as N increase • Contradiction ? • Theorem assumes the rate of each constituent flow becomes smaller as N increases, i.e., the rate of aggregate is constant independent of N

40. Background and related work • Wavelet-based multi-resolution analysis • Sub-RTT ON/OFF behavior due to TCP self-clocking • Effects of aggregation • Case study: the burstiness of an OC-48 trace • Smoothing effect of TCP pacing • Queueing performance

41. Queueing performance • Large buffer vs. small buffer • Heavy-load condition vs. moderate-load condition • Setup • Randomly sample input OC-48 trace to achieve desired utilization

42. 95-th percentile of queue size (Buffer=10MB) • Sub-RTT burstiness matters in moderate loading conditions • In heavy-load conditions, LRD is more important

43. Loss rate vs. buffer size (utilization=0.95) • Loss rate with pacing is significantly lower for small buffer size

44. Sub-RTT burstiness matters even in heavy-load conditions for underbuffered link Loss rate vs. offered load (Buffer=50KB)

45. TCP flow: Cp = C = 100Mbps, L=1500B, 4.5sec, 1000pkts • About 80% of interarrivals are within a factor of two from L/C

46. Most interarrivals at L/Cp and at (2L/C - L/Cp)

47. TCP flow: L=1500B, 37sec, 4200pkts • Cp = 100Mbps, C = 1.3Mbps (notice two interarrival modes)

49. TCP flow: L=1500B, 25sec, 5400pkts • Cp=100Mbps (note 50% of interarrivals at L/Cp)