Self-Similar through High-Variability: Statistical Analysis of Ethernet LAN Traffic at the Source Level

1. Self-Similar through High-Variability:Statistical Analysis of Ethernet LAN Traffic at the Source Level Walter Willinger, Murad S. Taqqu, Robert Sherman, Daniel V. Wilson Bellcore, Boston University SIGCOMM’95

2. Outline • Introduction • Self-similarity through high-variability • Ethernet LAN traffic measurements at the source level • Implications of the Noah Effect in practice • Conclusion

3. Introduction • Actual traffic exhibits correlations over a wide range of time scales (i.e. has long-range dependence). • Traditional traffic models focus on a very limited range of time scales and are thus short-range dependent in nature.

4. Introduction • Two problems that cause the resistance toward self-similar traffic modeling • What is a physical “explanation” for the observed self-similar nature of measured traffic from today’s packet networks? • What is the impact of self-similarity on network and protocol design and performance analysis?

5. Introduction • The superposition of many ON/OFF sources whose ON-periods and OFF-periods exhibit the Noah Effect produces aggregate network traffic that features the Joseph Effect. • Noah Effect: high variability or infinite variance • Joseph Effect: self-similar or long-range dependent

6. Self-Similarity through High-Variability • Idealized ON/OFF model • An ON-period can be followed by an ON-period and an OFF-period can succeed another OFF-period. • The distributions of the ON and OFF times may vary.

7. Idealized ON/OFF Model • Reward sequence {W(l ), l = 0,1,2,…} • {W(l )} is a 0/1-valued discrete time stochastic process. • W(l ) = 1 or 0 depends on whether or not there is a packet at time l. • {W(l )} consists of a sequence of 1’s (“ON-periods”) and 0’s (“OFF-periods”)

8. Idealized ON/OFF Model • The lengths of the ON- and OFF-periods are i.i.d. positive random variables, denoted Uk, k = 1,2,… • Let Sk = S0 + U2 + … + Uk , k  0 be the corresponding renewal times.

9. Idealized ON/OFF Model • Suppose there are M i.i.d. sources • The mth source has its own reward sequence {W(l ), l 0} • Superposition reward (“packet load”) b: non-overlapping time blocks j: the aggregation block number

10. Idealized ON/OFF Model • Suppose that U has a hyperbolic tail distribution, as M and b , adequately normalized is fractional Gaussian noise , which is self-similar with Hurst parameter ½  H <1

11. Idealized ON/OFF Model • Property (1) is the infinite variance syndrome or the Noah Effect. •   2 implies E(U2) =  •  > 1 ensures that E(U) < , and that S0 is not infinite

12. Idealized ON/OFF Model • Theorem 1. For large enough source Number M and Block aggregation size b, the cumulative load behaves statistically as where and . More precisely, where Llim means convergence in the sense of the finite-dimensional distributions (convergence in law)

13. Ethernet LAN Traffic Measurements at the Source Level • Location • Bellcore Morristown Research and Engineering Center • The first set • The busy hour of the August 1989 Ethernet LAN measurements • About 105 sources, 748 active source-destination pairs • 95% of the traffic was internal • The second set • 9 day-long measurement period in December 1994 • About 3,500 sources, 10,000 active pairs • Measurements are made up entirely of remote traffic

15. Textured Plots of Packet Arrival Times

16. Textured Plots of Packet Arrival Times

17. Checking for the Noah Effect • Complementary distribution plots • Hill’s estimate • Let U1, U2,…, Un denote the observed ON-(or OFF-)periods and write U(1) U(2) …U(n) for the corresponding order statistics

20. A Robustness Property of the Noah Effect • As far as the Noah Effect is concerned, it does not matter how the OFF-periods have been defined. • The similar investigation of sensitivity of the ON-period distributions to the choice of threshold value reveals the same appealing robustness feature of the Noah Effect.

23. Self-Similarity and the Noah Effect: 1989 Traffic Traces • 181(out of 748) source-destination pairs generated more than 93% of all the packets are considered. • The data at the source-destination level are consistent with • ON/OFF modeling assumption • Noah Effect for the distribution of ON/OFF-periods • -values for the ON- and OFF-periods may be different.

26. Self-Similarity and the Noah Effect: 1994 Traffic Traces • Non-Mbone traffic • 300 (out of 10,000) pairs responsible for 83% of the traffic are considered. • Self-similarity property of the aggregate packet stream is mainly due to the relative strong presence of the Noah Effect in the OFF-periods.

27. Self-Similarity and the Noah Effect: 1994 Traffic Traces • Mbone traffic • Only an analysis of the aggregate packet stream is performed. • The strong intensity of the Joseph Effect become obvious only after aggregation levels beyond 100ms. • There is no Noah Effect for ON-periods. • Reason: The use of unsophisticated compression algorithms resulted in packets bursts separated by comparatively large idle periods.

28. Traffic Modeling and Generation • Although network traffic is intrinsically complex, parsimonious modeling is still possible. • Estimating a single parameter  (intensity of the Noah Effect) is enough.

31. Performance and Protocol Analysis • The queue length distribution • Traditional (Markovian) traffic: decreases exponentially fast • Self-similar traffic: decreases much more slowly • Protocol design should be expected to take into account knowledge about network traffic such as the presence or absence of the Noah Effect.

32. Conclusion • The presence of the Noah Effect in measured Ethernet LAN traffic is confirmed. • The superposition of many ON/OFF models with Noah Effect results in aggregate packet streams that are consistent with measured network traffic, and exhibits the self-similar or fractal properties.