1 / 38

Scalable and Deterministic Overlay Network Diagnosis

Scalable and Deterministic Overlay Network Diagnosis. Yao Zhao, Yan Chen Northwestern Lab for Internet and Security Technology (LIST) Dept. of Computer Science Northwestern University http://list.cs.northwestern.edu. David Bindel Computer Science Division Dept. of EECS

carter
Download Presentation

Scalable and Deterministic Overlay Network Diagnosis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scalable and Deterministic Overlay Network Diagnosis Yao Zhao, Yan Chen Northwestern Lab for Internet and Security Technology (LIST) Dept. of Computer Science Northwestern University http://list.cs.northwestern.edu David Bindel Computer Science Division Dept. of EECS University of California at Berkeley

  2. When something breaks in the Internet, the Internet's very decentralized structure makes it hard to figure out what went wrong and even harder to assign responsibility. ̶̶ ̶ “Looking Over the Fence at Networks: A Neighbor's View of Networking Research”, by Committees on Research Horizons in Networking, National Research Council, 2001.

  3. Motivation • Internet diagnosis very important • To end users • To overlay network service providers (e.g., Akamai) • To Internet service providers (ISP) • But a very challenging problem due to the privacy of network administration • Solution • E2E measurements by end users -- overlay networks

  4. Related Works • Router based approaches [SOSP03] • Mostly ICMP based, ICMP rate limiting • Unscalable for simultaneous diagnosis • Cannot deterministically separate forward/backward path loss • Statistical approaches [MINC, INFOCOM03] • Non-deterministic: fundamentally under-constrained system • Inference based on temporal correlation in a multicast tree • Have to compromise for unicast, then sensitive to cross traffic • Optimization based on assumptions: # of lossy links small • Random sampling, linear programming, and Bayesian inference. • Unscalable: iterative refinement slow to converge for large networks

  5. Problem Formulation Given an overlay of N end hosts and O(N2) paths, to what granularity can we deterministically diagnosis the network fault? Assumptions: • Topology measurable • Can only measure the E2E path, not the link

  6. Outlines • Architecture and algebraic model • Identifying virtual links • Evaluation with simulations • Internet experiments

  7. Our Approach Overlay Network Operation Center Trouble spots location End hosts Diagnosis results: Qwest access link: 63.232.180.230->63.232.33.134 Peering between UUNET and AOL: 64.45.216.154->172.139.89.74 topology measurements • Monitor a basis set of O(n·logn)paths that fully describe the O(n2) paths • Decompose the paths into minimal deterministically identifiable segments • Compute the loss rate for each segment for diagnosis

  8. Linear algebraic model p1 Path loss rate p, link loss ratel 1 3 D C 2 B

  9. Putting All Paths Together = …

  10. Identifiable and Unidentifiable • Vectors in the row space of G are identifiable • Otherwise, unidentifiable x2 (1,1,0) A Row(path) space (identifiable) (1,-1,0) 1 3 p1 (1,1,1) D x1 p2 C 2 (0,0,1) B x3

  11. Outlines • Architecture and algebraic model • Identifying virtual links • Evaluation with simulations • Internet experiments

  12. Definition of Virtual Links • Uniquely identified shortest path segments • Identifiable • Consecutive • Undecomposable b 3’ 1 4 a e c 4’ 1’ 3 2 5 2’ d 4 paths, 5 links 5 virtual links

  13. One More Example 3’ 1 7 4 3 2’ 6’ 1’ 4’ 8 2 5 6 5’ • 6 paths, 8 links • 4 virtual links: • Corresponding to links 1, 2, 3+4+7 and 5+6+8 respectively

  14. Computing Virtual Links in Undirected Graph • Check if a vector is a virtual link • QR decomposition: • O(l·k) to check if a vector of length l is in row space of G • O(l2) potential virtual links in a path of length l • Total complexity O(l·k·l2·k)=O(l3·k2) x3 (1,1,1) (1,0,0) x1 (0,1,0) (1,1,0) x2 Row space • Small constant: only 4.2 sec for 135-node network

  15. Undirected vs. Directed Graphs • Directed graph • Any linear combination => Theorem: In a directed graph, no end-to-end path contains an identifiable subpath.

  16. Rescue: Good Path Algorithm • Identifying virtual links in undirected graphs • Use topology only • For directed graphs: additional info needed • Path loss rate • Use the link property constraint to break the deadlock • All the links in a good path are good links, i.e. no or little loss. • Most of the paths on the Internet are good paths

  17. System Flowchart • Monitors O(n·logn) paths that can fully describe all the O(n2) paths (SIGCOMM04) • Inherit load balancing, monitoring adaptation, etc. Optimization steps: find the minimal basis for identifiability test Good path algorithm on Select a basis of G’’: Reduced paths G’’ Select a basis of G, , for monitoring Measure topology to get G Find all lossy virtual links in G Estimatedloss rates for all paths in G Reduced paths G’ Good path algorithm on G Stage 1: set up scalablemonitoring system for diagnosis Stage 2: online update the measurements and diagnosis

  18. Outlines • Architecture and algebraic model • Identifying virtual links • Evaluation with simulations • Internet experiments

  19. Metrics • Avg length of lossy virtual links in all lossy paths • Diagnosis granularity • The avg number of potential lossy links in a lossy path • Example (Path 1 w/ lossy VL 1 of length 5, path 2 and 3 w/ lossy VL 2 of length 2) • Avg lossy VL length: (5+2)/2 = 3.5 • Avg diagnosis granularity: (5+2+2)/3 = 3 • Accuracy • Absolute error |p – p’ | • Relative error

  20. Simulation Methodology • Topology type • Three types of BRITE router-level topologies • Mecator topology • Topology size • 1000 ~ 20000 or 184k nodes • Fraction of end hosts on the overlay network • 10% ~ 50% • Link loss rate distribution • LLRD1 and LLRD2 models • Loss model • Bernoulli and Gilbert

  21. Sample of Simulation Results (Barabasi+Gilbert)

  22. Sample of Simulation Results (Barabasi+Gilbert)

  23. Results using Mercator Topology

  24. Gibbs Sampling (Infocom03) • D • Observed packet transmission and loss at the clients •  • Ensemble of loss rates of links in the network • Goal • Determine the posterior distribution P(|D) • Approach • Use Markov Chain Monte Carlo with Gibbs sampling to obtain samples from P(|D) • Draw conclusions based on the samples

  25. Comparison with Bayesian Inference using Gibbs Sampling (1)

  26. Comparison with Bayesian Inference using Gibbs Sampling (2)

  27. Outlines • Architecture and algebraic model • Identifying virtual links • Evaluation with simulations • Internet experiments

  28. Methodology • Planetlab • 135 end hosts • Topology measured by Traceroute • Avg path length is 17.2 • Path loss rate by active UDP probing • 300 40-byte UDP packets per measured path in 90 sec • Small overhead: 17.9kb if even measuring all paths

  29. Diagnosis Results The numbers in () are those after removing sequential link chains.

  30. Speed Results • On a Pentium-IV 3.2GHz PC • Average setup time (selecting 5,706 paths for monitoring): 109.3 seconds • Diagnosis of 2,858 lossy paths: 4.2 seconds

  31. Validation • Cross Validation • Divide 5720 paths into two sets (2860 each) • Get 571 virtual links from the first set • Check consistency with the second path set • 99.1% paths in the second set are consistent with virtual links computed by the first set.

  32. IP Spoofing based Validation • UDP: S:a, D:c, TTL=255 • UDP: S:a, D:b, TTL=255 • ICMP: S:r3, D:c, TTL=255 • UDP: S:c, D:b, TTL=2 c r1 a r3 r2 b

  33. IP Spoofing based Consistency Checking • Use the function of source routing of IP Spoofing to create new path segments • Validation is the same as cross validation • Results: • 1000 new path including part of segments in potential lossy paths • 94.1% loss spoofed paths are consistent with 361 out of 1664 lossy virtual links • 5.9% paths are inconsistent with 45 virtual links

  34. Conclusions • Propose the first deterministic and scalable overlay diagnosis system based on a linear algebraic approach • Diagnosis with virtual links: • Identifiable, consecutive and minimal path segments • Directed topology indecomposable to VL • Good path algorithms for rescue • Both simulation and Internet experiments show fast & accurate diagnosis w/ optimal granularity

  35. Backup Slides

  36. Previous Work • “Computing the unmeasured: An algebraic approach to Internet mapping,” INFOCOM’01 • Can’t work on directed graph • “User-level internet path diagnosis,” SOSP’03 • Need the support of routers • Not accurate • “Multicast-based inference of network-internal loss characteristics,” IEEE Transactions in Information Theory, 1999. • Multicast support or unicast approximation • “Server-based inference of Internet link lossiness,” INFOCOM'03 • Can only determine whether a link is lossy or not

  37. Distribution of Length of lossy Virtual Links

  38. IP Spoof Based Diagnosis

More Related