1 / 68

Interesting Links

Interesting Links. On the Self-Similar Nature of Ethernet Traffic Will E. Leland, Walter Willinger and Daniel V. Wilson BELLCORE Murad S. Taqqu BU. Analysis and Prediction of the Dynamic Behavior of Applications, Hosts, and Networks. Overview. What is Self Similarity?

livana
Download Presentation

Interesting Links

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interesting Links

  2. On the Self-Similar Nature of Ethernet TrafficWill E. Leland, Walter Willinger and Daniel V. Wilson BELLCOREMurad S. Taqqu BU Analysis and Prediction of the Dynamic Behavior of Applications, Hosts, and Networks

  3. Overview • What is Self Similarity? • Ethernet Traffic is Self-Similar • Source of Self Similarity • Implications of Self Similarity

  4. Section 1: What is Self-Similarity ?

  5. Intuition of Self-Similarity • Something “feels the same” regardless of scale (also called fractals)

  6. What is Self-Similarity? In case of stochastic objects like time-series, self-similarity is used in the distributional sense

  7. Pictorial View of Self-Similarity

  8. The Famous Data • Leland and Wilson collected hundreds of millions of Ethernet packets without loss and with recorded time-stamps accurate to within 100µs. • Data collected from several Ethernet LAN’s at the Bellcore Morristown Research and Engineering Center at different times over the course of approximately 4 years.

  9. Why is Self-Similarity Important? • Recently, network packet traffic has been identified as being self-similar. • Current network traffic modeling using Poisson distributing (etc.) does not take into account the self-similar nature of traffic. • This leads to inaccurate modeling which, when applied to a huge network like the Internet, can lead to huge financial losses.

  10. Problems with Current Models • A Poisson process • When observed on a fine time scale will appear bursty • When aggregated on a coarse time scale will flatten (smooth) to white noise • A Self-Similar (fractal) process • When aggregated over wide range of time scales will maintain its bursty characteristic

  11. Consequences of Self-Similarity • Traffic has similar statistical properties at a range of timescales: ms, secs, mins, hrs, days • Merging of traffic (as in a statistical multiplexer) does not result in smoothing of traffic Aggregation Bursty Data Streams Bursty Aggregate Streams

  12. Pictorial View of Current Modeling

  13. Side-by-side View

  14. Definitions and Properties • Long-range Dependence • autocorrelation decays slowly • Hurst Parameter • Developed by Harold Hurst (1965) • H is a measure of “burstiness” • also considered a measure of self-similarity • 0 < H < 1 • H increases as traffic increases

  15. Definitions and Properties Cont.’d • low, medium, and high traffic hours • as traffic increases, the Hurst parameter increases • i.e., traffic becomes more self-similar

  16. Properties of Self Similarity • X = (Xt : t = 0, 1, 2, ….) is covariance stationary random process (i.e. Cov(Xt,Xt+k) does not depend on t for all k) • Let X(m)={Xk(m)} denote the new process obtained by averaging the original series X in non-overlapping sub-blocks of size m. • Mean , variance 2 • Suppose that Autocorrelation Functionr(k)  k-β, 0<β<1 E.g. X(1)= 4,12,34,2,-6,18,21,35Then X(2)=8,18,6,28X(4)=13,17

  17. Auto-correlation Definition • X is exactly second-orderself-similar if • The aggregated processes have the same autocorrelation structure as X. i.e. • r (m) (k) = r(k), k0 for all m =1,2, … • X is [asymptotically] second-orderself-similar ifthe above holds when [ r (m) (k)  r(k), m  ] • Most striking feature of self-similarity: Correlation structures of the aggregated process do not degenerate as m  

  18. Traditional Models • This is in contrast to traditional models • Correlation structures of their aggregated processes degenerate as m   i.e. r (m) (k)  0 as m , for k = 1,2,3,... • Example: • Poisson Distribution • Self-Similar Distribution

  19. Long Range Dependence • Processes with Long Range Dependence are characterized by an autocorrelation function that decays hyperbolically as k increases • Important Property: This is also called non-summability of correlation

  20. Intuition • Short-range processes: • Exponential Decay of autocorrelations , i.e.: • r(k) ~ pk , as k , 0 < p < 1 • Summation is finite • The intuition behind long-range dependence: • While high-lag correlations are all individually small, their cumulative affect is important • Gives rise to features drastically different from conventional short-range dependent processes

  21. The Measure of Self-Similarity • Hurst Parameter H , 0.5 < H < 1 • Three approaches to estimate H (Based on properties of self-similar processes) • Variance Analysis of aggregated processes • Analysis of Rescaled Range (R/S) statistic for different block sizes • A Whittle Estimator

  22. Variance Analysis • Variance of aggregated processes decays as: • Var(X(m)) = am-b as m inf, • For short range dependent processes (e.g. Poisson Process), • Var(X(m)) = am-1 as m inf, • Plot Var(X(m)) against m on a log-log plot • Slope > -1 indicative of self-similarity

  23. The R/S statistic For a given set of observations, Rescaled Adjusted Range or R/S statistic is given by where

  24. Example • Xk = 14,1,3,5,10,3 • Mean = 36/6 = 6 • W1 =14-(1.6 )=8 • W2 =15-(2.6 )=3 • W3 =18-(3.6 )=0 • W4 =23-(4.6 )=-1 • W5 =33-(5.6 )=3 • W6 =36-(6.6 )=0 R/S = 1/S*[8-(-1)] = 9/S

  25. The Hurst Effect • For self-similar data, rescaled range or R/S statistic grows according to cnH • H = Hurst Paramater, > 0.5 • For short-range processes , • R/S statistic ~ dn0.5 • History: The Nile river • In the 1940-50’s, Harold Edwin Hurst studies the 800-year record of flooding along the Nile river. • (yearly minimum water level) • Finds long-range dependence.

  26. Whittle Estimator • Provides a confidence interval • Property: Any long range dependent process approaches FGN, when aggregated to a certain level • Test the aggregated observations to ensure that it has converged to the normal distribution

  27. Recap • Self-similarity manifests itself in several equivalent fashions: • Non-degenerate autocorrelations • Slowly decaying variance • Long range dependence • Hurst effect

  28. Section 2: Ethernet Traffic is Self-Similar

  29. Plots Showing Self-Similarity (Ⅰ) H=1 H=0.5 H=0.5 Estimate H  0.8

  30. Plots Showing Self-Similarity (Ⅱ) High Traffic 5.0%-30.7% Mid Traffic 3.4%-18.4% Low Traffic 1.3%-10.4% Higher Traffic, Higher H

  31. H : A Function of Network Utilization • Observation shows “contrary to Poisson” • Network Utilization H • As we shall see shortly, H measures traffic burstiness As number of Ethernet users increases, the resulting aggregate traffic becomes burstier instead of smoother

  32. Difference in low traffic H values • Pre-1990: host-to-host workgroup traffic • Post-1990: Router-to-router traffic • Low period router-to-router traffic consists mostly of machine-generated packets • Tend to form a smoother arrival stream, than low period host-to-host traffic

  33. H : Measuring “Burstiness” • Intuitive explanation using M/G/ Model • As α 1, service time is more variable, easier to generate burst • Increasing H !

  34. Summary • Ethernet LAN traffic is statistically self-similar • H : the degree of self-similarity • H : a function of utilization • H : a measure of “burstiness” • Models like Poisson are not able to capture self-similarity

  35. Discussions • How to explain self-similarity ? • Heavy tailed file sizes • How this would impact existing performance? • Limited effectiveness of buffering • Effectiveness of FEC • How to adapt to self-similarity? • Prediction • Adaptive FEC

  36. Section 3: Explaining Self - Similarity

  37. Introduction

  38. Introduction • The superposition of many ON/OFF sources whose ON-periods and OFF-periods exhibit the Noah Effect produces aggregate network traffic that features the Joseph Effect. • Noah Effect: high variability or infinite variance • Joseph Effect: self-similar or long-range dependent Also known as packet train models

  39. The Noah Effect • Noah Effect is the essential point of departure from traditional to self-similar traffic modeling • Results in highly variable ON-OFF periods : Train length and inter-train distances can be very large with non-negligible probabilities • Infinite Variance Syndrome : Many naturally occurring phenomenon can be well described with infinite variance distributions • Heavy-tail distributions,  parameter

  40. Existing Models • Traditional traffic models: finite variance ON/OFF source models • Superposition of such sourcesbehaves like white noise, with only short range correlations

  41. Idealized ON/OFF Model • Lengths of ON- and OFF periods are iid positive random variables, Uk • Suppose that U has a hyperbolic tail distribution, • Property (1) is the infinite variance syndrome or the Noah Effect. •   2 implies E(U2) =  •  > 1 ensures that E(U) < , and that S0 is not infinite

  42. http://statistik.wu-wien.ac.at/cgi-bin/anuran.pl

More Related