NetQuest: A Flexible Framework for Internet Measurement

1. NetQuest: A Flexible Framework for Internet Measurement Lili Qiu Joint work with Mike Dahlin, Harrick Vin, and Yin Zhang UT Austin

2. Motivation Server Sprint Server C&W AOL AT&T UUNet SBC Qwest Earthlink Server Server

3. Why is it so slow? Motivation (Cont.) AOL C&W Sprint AT&T UUNet SBC Qwest Earthlink

4. Motivation (Cont.) Applications are performance-aware • Server selection • Fault diagnosis • Traffic engineering • Overlay networks • Peer-to-peer applications • … Internet: large & decentralized Network measurement is important to • ISPs • Enterprise and university networks • Application and protocol designers • End users • …

5. Key Requirements • Scalable: work for large networks (100 –10000 nodes) • Flexible: accommodate different applications • Multi-user design • Multiple users interested in different parts of network or have different objective functions • Augmented design • Conduct additional experiments given existing observations, e.g., after measurement failures • Differentiated design • Different quantities have different importance, e.g., a subset of paths belong to a major customer Q: Which measurements to conduct to estimate the quantities of interest?

6. What We Want A function f(x) of link performance x • We use a linear function f(x)=F*x in this talk • Ex. 1: average link delay f(x) = (x1+…+x11)/11 • Ex. 2: end-to-end delays • Apply to any additive metric, eg. Log (1 – loss rate) x2 3 2 x4 x1 x3 x5 x6 4 5 1 x10 x7 x8 x11 7 6 x9

7. Problem Formulation What we can measure: e2e performance Network inference • Given e2e performance, infer link performance • Infer x based on y=F*x, y, and F Design of measurement experiments • State of the art • Probe every path (e.g., RON) • Rank-based approach [sigcomm04] • Select a “best” subset of paths to probe so that we can accurately infer f(x) • How to quantify goodness of a subset of paths?

8. Bayesian Experimental Design • Notations • D: a measurement design (eg., a subset of paths to probe) • I: an inference algorithm • U(D,I): utility function for design D and inference I • A good design maximizes the expected utility under the optimal inference algorithm

9. Design Criteria • Let , where is covariance matrix of x • Bayesian A-optimality • Goal: minimize the squared error • Bayesian G*-optimality • Goal: minimize the worst-case squared error • Bayesian D-optimal • Goal: maximize the expected gain in Shannon information

10. Flexibility Multi-user design • New design criteria: a linear combination of different users’ design criteria Augmented design • Ensure the newly selected paths in conjunction with previously monitored paths maximize the utility Differentiated design • Give higher weights to the important rowsof matrix F

11. Evaluation Methodology Data sets • NLANR traces • RTT, loss, traceroute measurements between pairs of 140 universities in Oct. 2004 • Resilient overlay network (RON) • RTT and loss among 12-15 hosts in March & May 2001 Accuracy metric

12. Evaluation Results (Cont.) All pairwise Rank-based

13. Evaluation Results (Cont.) All pairwise Rank-based

14. Summary of Other Results • Bayesian experimental design can support • Multi-user design • Augmented design • Differentiated design • Inference accuracy also depends on • Inference algorithms • Prior information

15. Summary Our contributions • Bring Bayesian experimental design to network measurement • Develop a flexible framework to accommodate different design requirements • Experimentally show its effectiveness On-going work • Build a toolkit • Gain operational experience • Develop applications • anomaly detection • performance knowledge plane

16. Thank you!