D ARRYL V EITCH The University of Melbourne

1. ACTIVE MEASUREMENT IN NETWORKS DARRYLVEITCH The University of Melbourne MetroGrid Workshop GridNets 2007 19 October 2007

2. A RENAISSANCE IN NETWORKMEASUREMENT • Not just monitoring • Traffic patterns: link, path, node, applications, management • Quality of Service: delay, loss, reliability • Protocol dynamics: TCP, VoIP, .. • Network infrastructure: routing, security, DNS, bottlenecks, latency.. • Knowing, understanding, improving network performance • Data centric view of networking • Must arise from real problems, or observations on data • Abstractions based on data • Results validated by data • In fact: the scientific method in networking • Not just getting numbers, Discovery

3. THE RISE OFMEASUREMENT • Papers between 1966-87 ( P.F. Pawlita, ITC-12, Italy ) • Queueing theory: several thousand • Traffic measurement: around 50 • Now have dedicated conferences: • Passive and Active Measurement Conference (PAM) 2000 … • ACM Internet Measurement Conference (IMC) 2001 ...

4. ACTIVEVERSUSPASSIVE MEASUREMENT • Typical Passive Aims • “At-a-point” or “Network Core” • Link utilisation, Link traffic patterns, Server workloads • Long term monitoring: Dimensioning, Capacity Planning, Source modelling • Engineering view: Network performance • Typical Active Aims • “End-to-End” or “Network edge” • End-to-End Loss, Delay, Connectivity, “Discovery”….. • Long/short term monitoring: Network health; Route state • Internet view: Application performance and robustness

6. ACTIVEPROBING VERSUS NETWORK TOMOGRAPHY • Active Probing • Typically over a single path • Use tandem FIFO queue model • Exploit discrete packet effects in semi-heuristic queueing analysis • Typical metrics: link capacities, available bandwidth • Network Tomography • Typically “network wide”: multiple destinations and/or sources • Simple black box node/link models, strong assumptions • Classical inference with twists • Typical metrics: per link/path loss/delay/throughput

7. TWO ENTREES • Loss Tomography Removing the temporal independence assumption Arya, Duffield, Veitch, 2007 • Active Probing Towards (rigorous) optimal probing Baccelli, Machiraju, Veitch, Bolot, 2007

8. TOMO - GRAPHY Tomos section Graphia writing

9. EXAMPLES OF TOMOGRAPHY • Atom probe tomography (APT) • Computed tomography (CT) (formerly CAT) • Cryo-electron tomography (Cryo-ET) • Electrical impedance tomography (EIT) • Magnetic resonance tomography (MRT) • Optical coherence tomography (OCT) • Positron emission tomography (PET) • Quantum tomography • Single photon emission computed tomography (SPECT) • Seismic tomography • X-ray tomography

10. COMPUTED TOMOGRAPHY

11. NETWORK TOMOGRAPHY Began with Vardi [1996] “Network Tomography: estimating source-destination traffic intensities from link data” Classes of Inversion Problems • End-to-end measurements  internal metrics • Internal measurements  path metrics The Metrics • Link Traffic (volume, variance) (Traffic Matrix estimation) • Link Loss (average, temporal) • Link Delay (variance, distribution) • Link Topology • Path Properties (network `kriging’) • Joint problems (use loss or delay to infer topology)

12. THE EARLY LITERATURE (INCOMPLETE!) • Traffic Matrix Tomography • AT&T ( Zhang, Roughan, Donoho et al. ) • Sprint ATL ( Nucci, Taft et al. ) • Loss/Delay/Topology Tomography • AT&T ( Duffield, Horowitz, Lo Presti, Towsley et al. ) • Rice ( Coates, Nowak et al. ) • Evolution: • loss, delay  topology • Exact MLE  EM MLE  Heuristics • Multicast  Unicast (striping )

13. 0 1 1 1 1 0 1 1 1 0 1 0 LOSS TOMOGRAPHY USING MULTICAST PROBING multicast probes ... Loss rates in logical Tree Loss Estimator ...

14. THE LOSS MODEL Stochastic loss process on link acts deterministically on probes arriving to Node and Link Processes • loss process on link • probe `bookkeeping’ process for node 1 1 … 0 0 1 1 0 0 … Link loss process 1 … 0 0 0 Bookkeeping process

15. ADDING PROBABILITY: LOSS DEPENDENCIES Independent Probes • Spatial: loss processes on different links independent • Temporal: losses within each link independent Model reduces to a single parameter per link, the passage or transmission probabilities

16. FROM LINK PASSAGE TO PATH PASSAGE PROBABILITIES Path probabilities: only ancestors matter Sufficient to estimate path probabilities

17. ACCESSING INTERNAL PATHS Estimation: joint path passage probability, single probe Aim: estimate Obtain a quadratic in 0 1 1 1 1 0 Original MINC loss estimator for binary tree [Cáceres,Duffield, Horowitz, Towsley 1999] 1 1 1 1 1 1 0 0 0 … … … 10 / 19

18. OBTAIN PATHS RECURSIVELY Estimation: joint path passage probability, single probe 10 / 19

19. TEMPORAL INDEPENDENCE: HOW FAR TO RELAX ? Before • Spatial: link loss processes independent • Temporal: link loss processes Bernoulli • Parameters: link passage probabilities After • Spatial: link loss processes independent • Temporal: link loss processes stationary, ergodic • Parameters: joint link passage probabilities over index sets Full characterisation/identification possible!

20. TARGET LOSS CHARACTERISTIC • Loss run-length distribution ( density, mean ) Pr 1 2 3 … Loss-run length … … Loss/pass bursts • Importance • Impacts delay sensitive applications like VoIP (FEC tuning) • Characterizes bottleneck links

21. ACCESSING TEMPORAL PARAMETERS: GENERAL PROPERTIES • Sufficiency of joint link passage probabilities e.g. • Mean Loss-run length • Loss-run distribution 7

22. JOINT PASSAGE PROBABILITIES • Joint link passage probability • Joint path passage probability

23. ESTIMATION: JOINT PATH PASSAGE PROBABILITY No of equations equal to degree of node k computed recursively over index sets = 0 for

24. ... ... ESTIMATION: JOINT PATH PASSAGE PROBABILITY • Estimation of in general trees • Requires solving polynomials with degree equal to the degree of node k • Numerical computations for trees with large degree • Recursion over smaller index sets • Simpler variants • Subtree-partitioning • Requires solutions to only linear or quadratic equations • No loss of samples • Also simplifies existing MINC estimators • Avoid recursion over index sets by considering only • subsets of receiver events which imply

25. SIMULATION EXPERIMENTS • Setup • Loss process • Discrete-time Markov chains • On-off processes: pass-runs geometric, loss-runs truncated Zipf • Estimation • Passage probability • Joint passage probability for a pair of consecutive probes • Mean loss-run length: • Relative error:

26. EXPERIMENTS • Estimation for shared path in case of two-receiver binary trees Markov chain On-off process

27. EXPERIMENTS • Estimation for shared path in case of two-receiver binary trees Markov chain On-off process

28. EXPERIMENTS • Estimation of for larger trees Link 1 • Trees taken from router-level map of AT&T network produced by Rocketfuel • (2253 links, 731 nodes) • Random shortest path multicast trees with • 32 receivers. Degree of internal nodes from 2 to 6, maximum height 6

29. VARIANCE • Estimation for shared path in case of tertiary tree Standard errors shown for various temporal estimators

30. CONCLUSIONS • Estimators for temporal loss parameters, in addition to loss rates • Estimation of any joint probability possible for a pattern of probes • Class of estimators to reduce computational burden • Subtree-partition: simplifies existing MINC estimators • Future work • Asymptotic variance • MLE for special cases (Markov chains) • Hypothesis tests • Experiments with real traffic

31. OPTIMAL PROBING IN CONVEXNETWORKS FRANÇOIS BACCELLI, SRIDHAR MACHIRAJU, DARRYL VEITCH, JEAN BOLOT

32. PREVIOUS WORK • The Case Against Poisson Probing • Zero-sampling bias not unique to Poisson (in non-intrusive case) • PASTA talks about bias, is silent on variance • PASTA is not optimal for variance or MSE • PASTA is about sampling only, is silent on inversion • Probe Pattern Separation Rule: select inter-probe (or probe pattern) • separations as i.i.d. positive random variables, bounded above zero. • Example: Uniform (i.i.d.) separations on [ 0.9μ, 1.1μ ] • Aims for variance reduction • Avoids phase locking (leads to sample path `bias’) • Allows freedom of probe stream design 10

33. LIMITATIONS • Results derived in context of delay only • No optimality result (expected to be highly system and traffic dependent) 10

34. NEW WORK • Extended all results to loss case (loss and delay in uniform framework) • For convex networks, have universal optimum for variance • But: sample-path bias • Give family of probing strategies with • Zero bias and strong consistency • Variance as close to optimal as desired (tunable) • Fast simulation • Consistent with spirit of Separation Rule • But are networks convex? • Some systems when answer is known to be Yes • virtual work of M/G/1 • loss process of M/M/1/K • Insight into when No • Real data when answer seems to be Yes 10

35. TWO PROBLEMS: SAMPLING AND INVERSION • Sampling • For end-to-end delay of a probe of size x • Only have probe samples • Inversion • From the measured delay data of perturbed network, may want: • Unperturbed delays • Link capacities • Available bandwidth • Cross traffic parameters at hop 3 • TCP fairness metric at hop 5 …. Here we focus on sampling only, do so using non-intrusive probing 10

36. THE QUESTION OF GROUND TRUTH • Non-intrusive Delay • Delay process using zero sized probes still meaningful • Each probe carries a delay: samples available • Non-intrusive Loss • Losses with x=0 are hard to find.. meaningful but useless • Lost probes are not available (where? How?) • Probes which are not lost tell us ? about loss? Cannot base non-intrusive probing on virtual (x=0) probes 10

37. GENERAL GROUND TRUTH • Approach • Define end-to-end process directly in continuous time • Must be a function of system in equilibrium - no probes, no perturbation! • Non-intrusive probing defined directly as a sampling of this process: • Interpretation: what probe would have seen if sent in at time . • Note • Process may be very general, not just isolated probes but probe patterns! • Applies equally to loss or delay • This is the only way, even virtual probes can be intrusive! 10

38. LOSS GROUND TRUTH EXAMPLES • 1-hop • FIFO, buffer size K bytes, droptail • If packet based instead, becomes independent of x ! • 2-hop • Depends on x • Other examples: • Packet pair jitter • Indicator of packet loss in a train • Largest jitter in a chain, or number lost 10

39. NIPSTA: NON-INTRUSIVE PROBING SEES TIME AVERAGES • Strong Consistency: • Empirical averages seen by probe samples converge to true expectation • of continuous time process, assuming: • Stationarity and ergodicity of CT • Stationarity and ergodicity of PT (easy) • Independence of CT and PT (easy) • Joint ergodicity between CT and PT Result follows just as in delay work, using marked point processes, Palm Calculus and Ergodic Theory. • Not Restricted to Means : • Temporal quantities also, eg jitter • Probe-train sampling strategies 10

40. OPTIMAL VARIANCE OF SAMPLE MEAN Covariance function Sample Mean Estimator Periodic Probing 10

41. PERIODIC PROBING IS OPTIMAL ! Compare integral terms : If R convex, then can use Jensen’s inequality: No amount of probe train design can beat periodic! • Problem : • Periodic is not mixing, so joint ergodicity not assured (phase lock) • If occurs, get `sample path bias’ (estimator not strongly consistent) 10

42. GAMMA RENEWAL PROBING Gamma Law • Gamma Renewal Family: • As increases, variance drops at constant mean • Poisson is beaten once • Process tends to periodic as • Sensitivity to periodicities increases however • Proof: • Again uses Jensen’s inequality • Uses a conditioning trick and a technical result on Gamma densities 10

43. TEST WITH REAL DATA FOR MEAN DELAY 10

44. EXAMPLE WITH REAL DATA 10

45. EXAMPLE WITH REAL DATA 10

46. CONCLUSION TO A MORNING • Active measurement continues to develop! • Grid applications will no doubt lead to interesting new problems Merci de votre attention