Multiscale Traffic Processing Techniques for Network Inference and Control

1. Multiscale Traffic Processing Techniques for Network Inference and Control R. Baraniuk R. Nowak E. Knightly R. Riedi V. Ribeiro S. Sarvotham A. Keshavarz Rice University NMS PI meeting Chicago November 2002 Signal Processing in Networking Rice University, SPiN Group spin.rice.edu

2. Objective: Reduced complexity, multiscale link models with known accuracy Innovative Ideas Multifractal analysis Multiplicative modeling Multiscale queuing Chirps for probing Impact Congestion control performance improvement Dynamical streaming monitoring/anomaly detection Rice University, SPiN Group spin.rice.edu

3. Chirp Probingalways-on, non-intrusiveon-line decisionanomaly detection Effort 1 Rice University, SPiN Group spin.rice.edu

4. Efficient probing: PathChirp • Traditional probing paradigm: • Produce (light) congestion • PacketPair: • Sample the traffic • Pathload: flood at variable rate • intolerable level of congestion • TOPP: • PacketPairs at variable spacing • New: • PathChirp: • Variable rate within a train of probes • More efficient, light Rice University, SPiN Group spin.rice.edu

5. PathChirp Performance • Real world tool available • performs comparably to • PathLoad • PacketPair • TOPP • …at smaller probing rate • …more robust to bursty traffic conditions • Ongoing work: • Exploit dispersion information to become robust against multiple hops Rice University, SPiN Group spin.rice.edu

6. 30% util Non-intrusive: Queue Samplers Error against scale Traffic error • Traffic arrival process and Queue process equivalent • Advantages of sampling the Queue • Slim probe packets • smaller probing rate or higher accuracy • Robust to Multi-Hop • Delay simply accumulates while • …probe dispersion is vulnerable (pair-compression at fast links) • Delphi: • further advantage through • Model-based inference (MSQ) • Overcomes uncertainty principle • Ongoing work: • optimal probing pattern • Uniform / Back2Back / variable rate • optimal probing paradigm • Traffic samplers / Queue samplers / Delphi • Integration into network simulators Queue error Traffic error Queue error 50% util Traffic error Queue error 80% util Delphi error Rice University, SPiN Group spin.rice.edu

7. Multiple Hops: Collaboration • Monitor packets at intermediate point of path • IP-tunneling • Coordinated Measuremens (IPEX: Krishnan) • Integrating probing tools into simulators • JavaSim (UIUC: Hou) • NS-2 (ISI: Heidemann) • Probing buffer at core router • Passive inference (Sprint Labs: Moon) Rice University, SPiN Group spin.rice.edu

8. Connection-level Analysis and Modeling of Network Trafficunderstanding the cause of burstscontrol and improve performancedetect changes of network state Effort 2 Rice University, SPiN Group spin.rice.edu

9. 99% Mean Non-Gaussianity and Dominance Systematic study: time series separation • For each bin of 500 ms: remove packets of the ONEstrongest connection • Leaves “Gaussian” residual traffic = + Overall traffic 1 Strongest connection Residual traffic Rice University, SPiN Group spin.rice.edu

10. Origins of Alpha Traffic • Mostly TCP • flow & congestion controlled • HTTP, SMTP, NNTP, etc. • Several gateways, Abilene Core • Alpha connections tend to have the same source-destination IP addresses. • Any large volume connection with the same e2e IP addresses as an alpha connection is also alpha.  Systematic cause of alpha: Large file transfers over high bandwidth e2e paths. Rice University, SPiN Group spin.rice.edu

11. Bursts arise from large transfers over fast links. Simple Connection Taxonomy This is the only systematic reason Rice University, SPiN Group spin.rice.edu

12. 5 2 10 10 Beta Beta Alpha Alpha 4 1 10 10 cwnd (B) 3 0 10 10 2 -1 10 10 3 4 5 6 3 4 5 6 10 10 10 10 10 10 10 10 peak-rate (Bps) Correlation coefficient=0.01 Cwnd or RTT? Colorado State University trace, 300,000 packets 1/RTT (1/s) peak-rate (Bps) Correlation coefficient=0.68 RTT has strong influence on bandwidth and dominance. Rice University, SPiN Group spin.rice.edu

13. Examples of Alpha/Beta Connections Alpha connections burst because of short round trip time, not large TCP window Appear to be flow (not congestion) controlled Rice University, SPiN Group spin.rice.edu

14. Alpha traffic: Flow or Congestion Control? • Alpha connection show • unusually small advertised window • drastic drop in advertised window (to zero) • …which correlates with burst arrival • Alpha connections with same END host • Share same high peak rate • End-host becomes bottleneck, network does not control • TCP is known to be unfair to long RTT connections • TCP appears to foster also the Alpha bursts • What determines the alpha-peak rates? Rice University, SPiN Group spin.rice.edu

15. Modeling Alpha Traffic • ON/OFF model revisited: High variability in connection rates (RTTs) Low rate = beta High rate = alpha + + + = = stableLevy noise fractionalGaussian noise

16. Impact: Network Simulation Realistic: heterogeneous end-to-end bandwidth Simple: equal bandwidth • Simulation: ns topology to include alpha links • Congestion control • Design and management Rice University, SPiN Group spin.rice.edu

17. Impact: Traffic simulation • ClassicalON/OFF model captures only • Gaussian Beta-component • Large scale fluctuations • Variable rate ON/OFF captures also • burstiness Realistic traffic contains ON/OFF sources with heterogeneous rates Rice University, SPiN Group spin.rice.edu

18. Ongoing work: simulation • Relate physical network parameters to traffic dynamics • File size distributions • Network topology / RTT distribution • Network devices (links, servers) • Congestion control vs flow control • Predict performance from simple set of parameters • Integration into network simulators to abstract network clouds Rice University, SPiN Group spin.rice.edu

19. Impact: Performance • Beta Traffic rules the small Queues • Alpha Traffic causes the large Queue-sizes (despite small Window Size) All Alpha Packets Queue-size with Alpha Peaks Rice University, SPiN Group spin.rice.edu

20. Ongoing Work • Total true Queue: • Q = Qa+Qb (contributions from alpha and beta component) • Model assumption: • Alpha=stable, Beta=fractional Brownian motion • Optimal setup of two individual Queues driven by alpha and beta components to come closest to Qa and Qb • quantify impact of components on queuing Rice University, SPiN Group spin.rice.edu

21. Separation on Connection Level • Alpha connections: dominant. Properties: • Few, light load • Responsible for violent bursts • Responsible for large queuing delays • Peak rate > mean arrival rate + 1 std dev • Typically short RTT • Typically FLOW-CONTROLLED (limited at receiver) • Beta connections: Residual traffic • Main load • Gaussian, LRD • Typically bandlimited at bottleneck link • C+ analyzer (order of sec) Rice University, SPiN Group spin.rice.edu

22. Summary: Goals • Network/user-driven traffic model • Obtain performance parameters directly from network and user specifications • Non-intrusive, always-on, global traffic estimation • Anomaly detection in network (change in network state) Collaborations & Tech Transfer • Monitoring tool using chirps (Caida, SLAC, Sprint [students]) • Verification/”internal” measurements using • IP-tunneling, coordinated measurements (IPEX) • Integration of PathChirp into network simulators (UIUC, ISI) • Online traffic probing and analysis tools for • Integrated demo (GaTech) • Demystify self-similarity (UC Riverside) Rice University, SPiN Group spin.rice.edu

23. spin.rice.edu Rice University, SPiN Group spin.rice.edu