Multiscale Traffic Processing Techniques for Network Inference and Control

1. Multiscale Traffic Processing Techniques for Network Inference and Control R. Baraniuk R. Nowak E. Knightly R. Riedi V. Ribeiro S. Sarvotham A. Keshavarz NMS PI meeting San Diego May 2003 SPiN.Rice.edu: Signal Processing in Networking

2. Chirp Probingalways-on, non-intrusivebandwidth estimationon-line decisionanomaly detection Effort 1 Rice University, SPiN Group spin.rice.edu

3. Milestones pathChirp May 2003 milestone: • C code testing + distribution [completed] • Allen McIntosh (Telcordia): testing, useful comments, • SPAWAR (hosted Phuong Nguyen at Rice), • CAIDA (kc Claffy, Margaret Murray), • RPI (Shivkumar). • Demo: on-line estimation of bandwidth Towards Nov 2003 milestones: • Integration of pathChirp • GaTech pdns (Riley/Fujimoto) [completed] • UIUC, JavaSim (Hou) [in progress] • Validation in controllable environment [enabled, to be done] Towards May 2004 milestones: [basis provided, work to be done] • Final tool and theory for bursty traffic over multiple hops Rice University, SPiN Group spin.rice.edu

4. Efficient probing: PathChirp • Traditional probing paradigm: • Produce (light) congestion • PacketPair: • Sample the traffic • Pathload: flood at variable rate • intolerable level of congestion • TOPP: • PacketPairs at variable spacing • New: • PathChirp: • Variable rate within a train of probes • More efficient, light Rice University, SPiN Group spin.rice.edu

5. Methodology Departure pattern Queuing against departure pathChirp Developments pathChirp • …a real world tool • …with improved performance • Increased queuing delay correlates with cross traffic on network path • Last excursion in chirp  link capacity • Weighted averaged of onset of excursions available link resources Rice University, SPiN Group spin.rice.edu

6. Methodology Departure pattern Queuing against departure Real world experiments Number of chirps  Estimation against true x-traffic Internet experiment 12 chirps pathChirp Developments pathChirp • …a real world tool • …with improved performance • Queuing delay  cross traffic • Final excursion  link capacity • Averaged excursions available resources • …converges in a handful of RTTs Rice University, SPiN Group spin.rice.edu

7. PathChirp Performance PathLoad converged after 6.7 Mb pathChirp • performs comparably to • PathLoad • PacketPair • TOPP • …at smaller probing rate • …more robust to bursty traffic • Best paper at PAM2003 • Ongoing work: • Exploit dispersion information captured in excursions to become robust against multiple hops .5 Mb 1 Mb pathChirp vs TOPP square error Rice University, SPiN Group spin.rice.edu

8. Connection-level Analysis and Modeling of Network Trafficunderstanding the cause of burstscontrol and improve performancedetect changes of network state Effort 2 Rice University, SPiN Group spin.rice.edu

9. Milestones Alpha-Beta May 2003 milestone [completed]: • C++ version of decomposition and analysis module Towards Nov 2003 milestone: • Verification of alpha-beta hypothesis in wider range of topologies, protocols, applications [Analysis module ready; collection and analysis to be done] • Collaborations with Telcordia and SLAC [initiated] • Collaborations with CAIDA [pursuing] December 2004 milestones [to be done]: • Integration into simulators, verification in large simulation • Applications: alpha-bottleneck aware AQM, Admission control Rice University, SPiN Group spin.rice.edu

10. 99% Mean Non-Gaussianity and Dominance Connection level separation: • remove packets of the ONEstrongest connection • Leaves “Gaussian” residual traffic Traffic components: • Alpha connections: high rate (> ½ bandwidth) • Beta connections: all the rest = + Overall traffic 1 Strongest connection Residual traffic Rice University, SPiN Group spin.rice.edu

11. Bursts arise from large transfers over fast links. Simple Connection Taxonomy Rice University, SPiN Group spin.rice.edu

12. 5 2 10 10 Beta Beta Alpha Alpha 4 1 10 10 cwnd (B) 3 0 10 10 2 -1 10 10 3 4 5 6 3 4 5 6 10 10 10 10 10 10 10 10 peak-rate (Bps) Correlation coefficient=0.01 CWND or RTT? Colorado State University trace, 300,000 packets 1/RTT (1/s) peak-rate (Bps) Correlation coefficient=0.68 Short RTT correlates directly with high rate and bursts. Rice University, SPiN Group spin.rice.edu

13. Impact: Performance • Beta Traffic rules the small Queues • Alpha Traffic causes the large Queue-sizes (despite small Window Size) Queue-size overlapped with Alpha Peaks Total traffic Alpha connections Rice University, SPiN Group spin.rice.edu

14. Two models for alpha traffic Impact of alpha burst in two scenarios: Flow control at end hosts TCP advertised window Congestion control at router TCP congestion window Rice University, SPiN Group spin.rice.edu

15. Modeling Alpha Traffic • ON/OFF model revisited: High variability in connection rates (RTTs) Low rate = beta High rate = alpha + + + = = stableLevy noise fractionalGaussian noise

16. Alpha-Beta Model of Traffic • Model assumptions: • Total traffic = Alpha component + Beta component • Alpha and Beta are independent • Beta=fractional Brownian motion • Alpha traffic: two scenarios • Flow control through thin or busy end-hosts • ON-OFF Burst model • Congestion control allowing large CWND • Self-Similar Burst model • Methods of analysis • Self-similar traffic • Queue De-multiplexing • Variable service rate Rice University, SPiN Group spin.rice.edu

17. De-Multiplexing: Equal critical time-scales Q-tail Pareto Due to Levy noise Self-similar Burst Model • Alpha component = self-similar stable • (limit of a few ON-OFF sources in the limit of fast time) • This models heavy-tailed bursts • (heavy tailed files) • TCP control: alpha CWND arbitrarily large • (short RTT, future TCP mutants) • Analysis via De-Multiplexing: • Optimal setup of two individual Queues to come closest to aggregate Queue Beta (top) + Alpha

18. ON-OFF Burst Model • Alpha traffic = High rate ON-OFF source (truncated) • This models bi-modal bandwidth distribution • TCP: bottleneck is at the receiver (flow control through advertised window) • Current state of measured traffic • Analysis: de-multiplexing and variable rate queue • Queue-tail Weibull (unaffected) unless • rate of alpha traffic larger than • capacity – average beta arrival • and duration of alpha ON period heavy tailed Beta (top) + Alpha Variable Service Rate Rice University, SPiN Group spin.rice.edu

19. Alpha traffic: Influence of TCP • All Alpha connections show • Unusually small advertised window • Drastic drop in advertised window (sometimes to zero) • …which correlates with burst arrival •  Flow controlled, Weibull Q-tails Rice University, SPiN Group spin.rice.edu

20. Separation on Connection Level • Alpha connections: dominant. Properties: • Definition: Peak rate > mean arrival rate + 1 std dev • Few, light load • Responsible for violent bursts, large queuing delays • Typically short RTT • Typically FLOW-CONTROLLED (limited at receiver) • Beta connections: Residual traffic • Main load • Gaussian, LRD • Typically limited at bottleneck link • Future of empowered hosts and transfer protocols: • Higher peaks, larger bursts, longer queues Rice University, SPiN Group spin.rice.edu

21. Future work • a/b: Network/user-driven traffic model • Correlations between network and user • Through simulation and measurements assess impact of protocols, applications, clientele, end-host server • Performance parameters from network and user specifications • pathChirp • Model based estimation meeting challenges of bursty traffic • Through simulation validate realism (multihop, bursty traffic) • Anomaly detection through chirp-web Current Collaborations & Tech Transfer • IP-tunneling, coordinated measurements (Telcordia) • Integration of PathChirp into network simulators (GaTech, UIUC) • Ready for integration into SPAWAR • Demystify self-similarity (UC Riverside) Rice University, SPiN Group spin.rice.edu

22. spin.rice.edu Rice University, SPiN Group spin.rice.edu