Part III: Measuring Inter-domain Paths

1. Part III: Measuring Inter-domain Paths

2. Packet forwarding path Destination Internet IP traffic Source Forwarding path - the path packets traverse through the Internet from a source to a destination

3. An inter-domain level view AS D Destination Internet AS C IP traffic AS A AS B Source An IP forwarding path often span across multiple Autonomous Systems.

4. Why do we care? • Characterize end-to-end network paths • Diagnose routing anomalies • Discover Internet topology

5. Why do we care? • Characterize end-to-end network paths • Latency • Capacity • Link utilization • Loss rate. • Diagnose routing anomalies • Discover Internet topology

6. Varies link capacity Destination Internet Source

7. Different loss rate Destination Internet Source

8. Traffic engineering Destination Internet Source Customer service enhancement

9. Why do we care? • Characterize end-to-end network paths • Diagnose routing anomalies • Forwarding loop, black holes, routing changes, unexpected paths, main component of end-to-end latency. • Discover Internet topology

10. Forwarding loops Destination Internet Source

11. Black holes Destination Internet Source

12. Routing changes Destination Internet Source

13. Unexpected routes Destination Internet Source

14. Performance bottleneck Destination Internet Source

15. Why do we care? • Characterize end-to-end network paths • Diagnose routing anomalies • Discover Internet topology • Server placement

16. Internet topology Server Internet Client Client Client

17. Proxy Server placement Server Internet Client Client Client

18. Key challenge • Need to understand how packets flow through the Internet without real-time access to proprietary routing data from each domain. • Identify accurate packet forwarding paths • Characterize the performance metrics of each hop along the paths

19. Identify forwarding path • Traceroutegives IP level forwarding path • IP address of the router interfaces on a forwarding path • RTT statistics for each hop along the way

20. 1 169.229.62.1 2 169.229.59.225 3 128.32.255.169 4 128.32.0.249 5 128.32.0.66 6 209.247.159.109 7 * 8 64.159.1.46 9 209.247.9.170 10 66.185.138.33 11 * 12 66.185.136.17 13 64.236.16.52 inr-daedalus-0.CS.Berkeley.EDU soda-cr-1-1-soda-br-6-2 vlan242.inr-202-doecev.Berkeley.EDU gigE6-0-0.inr-666-doecev.Berkeley.EDU qsv-juniper--ucb-gw.calren2.net POS1-0.hsipaccess1.SanJose1.Level3.net ? ? pos8-0.hsa2.Atlanta2.Level3.net pop2-atm-P0-2.atdn.net ? pop1-atl-P4-0.atdn.net www4.cnn.com Traceroute from UC Berkeley to www.cnn.com Traceroute output: (hop number, IP address, DNS name) 1 169.229.62.1 2 169.229.59.225 3 128.32.255.169 4 128.32.0.249 5 128.32.0.66 6 209.247.159.109 7 * 8 64.159.1.46 9 209.247.9.170 10 66.185.138.33 11 * 12 66.185.136.17 13 64.236.16.52 inr-daedalus-0.CS.Berkeley.EDU soda-cr-1-1-soda-br-6-2 vlan242.inr-202-doecev.Berkeley.EDU gigE6-0-0.inr-666-doecev.Berkeley.EDU qsv-juniper--ucb-gw.calren2.net POS1-0.hsipaccess1.SanJose1.Level3.net ? ? pos8-0.hsa2.Atlanta2.Level3.net pop2-atm-P0-2.atdn.net ? pop1-atl-P4-0.atdn.net www4.cnn.com

21. traceroute to cnn.com (64.236.24.12), 30 hops max, 40 byte packets 1 oden (135.207.16.1) 1 ms 1 ms 1 ms 2 * * * 3 attlr-gate (192.20.225.1) 2 ms 2 ms 2 ms 4 12.119.155.157 (12.119.155.157) 3 ms 4 ms 4 ms 5 gbr6-p52.n54ny.ip.att.net (12.123.192.18) 4 ms 4 ms 4 ms 6 tbr2-p012401.n54ny.ip.att.net (12.122.11.29) 4 ms (ttl=249!) 5 ms (ttl=249!) 5 ms (ttl=249!) 7 ggr2-p390.n54ny.ip.att.net (12.123.3.62) 4 ms 5 ms 4 ms 8 att-gw.ny.aol.net (192.205.32.218) 4 ms 4 ms 4 ms 9 bb2-nye-P1-0.atdn.net (66.185.151.66) 4 ms 4 ms 4 ms 10 bb2-vie-P8-0.atdn.net (66.185.152.201) 13 ms (ttl=245!) 12 ms (ttl=245!) 12 ms (ttl=245!) 11 bb1-vie-P11-0.atdn.net (66.185.152.206) 10 ms 10 ms 10 ms 12 bb1-cha-P7-0.atdn.net (66.185.152.28) 20 ms 20 ms 20 ms 13 bb1-atm-P6-0.atdn.net (66.185.152.182) 25 ms 25 ms 25 ms 14 pop1-atl-P4-0.atdn.net (66.185.136.17) 25 ms (ttl=243!) 24 ms (ttl=243!) 24 ms (ttl=243!) 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * * 22 * * * 23 * * * 24 * * * 25 * * * 26 * * * 27 * * * 28 * * * 29 * * * 30 * * * Traceroute from AT&T Research to www.cnn.com Destination unreachable! Who is responsible for the forwarding problem?

22. Need to know Inter-domain level path AS D www.cnn.com Internet AS C AS A AS B AT&T Research Routing loop in AS C!

23. How to obtain AS level paths • BGP AS path • Traceroute AS path

24. AS C AS A AS B Prefix d Forwarding path: data traffic BGP AS path Signaling path: control traffic d: path=[BC] d: path=[C] Prefix AS path dA B C … … Is BGP AS path the answer? No!

25. BGP AS path is not the answer • Requires timely access to BGP data • Signaling path may differ from forwarding path • Route aggregation and filtering • Routing anomalies: e.g., deflections, loops [Griffin2002] • BGP misconfigurations: e.g., incorrect AS prepending Two paths may differ precisely when operators most need accurate data to diagnose a problem!

26. AS A AS B AS C AS D Traceroute AS path • Obtain IP level path using traceroute • Map IP addresses to ASes b c d e a Source Destination Is traceroute AS path the answer? NO!

27. AS25 AS25 AS25 AS25 AS11423 AS3356 AS3356 AS3356 AS3356 AS1668 AS1668 AS1668 AS5662 Berkeley Calren Level3 GNN CNN Example: UC Berkeley to CNN Traceroute output: (hop number, IP) 1 169.229.62.1 2 169.229.59.225 3 128.32.255.169 4 128.32.0.249 5 128.32.0.66 6 209.247.159.109 7 * 8 64.159.1.46 9 209.247.9.170 10 66.185.138.33 11 * 12 66.185.136.17 13 64.236.16.52

28. Traceroute AS path is not the answer • Identifying ASes along forwarding path is surprisingly difficult! • Internet route registry • Origin AS in BGP routes

29. Internet route registry • Whois database • E.g. NANOG traceroute, prtraceroute • Out-of-date, incomplete • Address allocation to customers • Acquisition, mergers, break-ups

30. Origin AS in BGP routes • Last AS in the AS path for each prefix • More accurate and complete than whois data

31. Limitations of BGP origin AS • Multiple Origin AS (MOAS) • Infrastructure addresses may not be advertised • Addresses announced by someone else

32. Limitations of BGP origin AS • Multiple Origin AS (MOAS) • Multi-homing • Misconfiguration • Internet eXchange Points (IXPs) • Infrastructure addresses may not be advertised • Addresses announced by someone else

33. Limitations of BGP origin AS • Multiple Origin AS (MOAS) • Infrastructure addresses may not be advertised • Does not require to be announced publicly • Security concerns • Addresses announced by someone else

34. Limitations of BGP origin AS • Multiple Origin AS (MOAS) • Infrastructure addresses may not be advertised • Addresses announced by someone else • Static routed customers • Shared equipments at boundary between ASes Need accurate IP-to-AS mapping!

35. Accurate AS-level traceroute Combine BGP and traceroute data to find a better answer!

36. Assumptions • IP-to-AS mapping • Mappings from BGP tables are mostly correct. • Change slowly • BGP paths and forwarding paths mostly match. • 70% of the BGP path and traceroute path match

37. BGP path and traceroute path could differ! • Inaccurate IP-to-AS mapping • Traceroute problems • Legitimate mismatches

38. BGP path and traceroute path could differ! • Inaccurate IP-to-AS mapping • Internet eXchange Points (IXPs) • Sibling ASes • Unannounced infrastructure addresses • Traceroute problems • Legitimate mismatches

39. Internet eXchange Points (IXPs) • Shared infrastructure connected to multiple service providers • Exchange BGP routes and data traffic • May have its own AS number or announced by participating ASes • Dedicated BGP sessions between pairs of participating ASes • E.g., Mae-East, Mae-West, PAIX.

40. IXPs cause extra AS hop • Extra AS hop in traceroute path • Large number of fan-in and fan-out ASes • Non-transit AS, small address block, likely MOAS

41. IXPs cause extra AS hop E A A E F B F B D G C G C Traceroute AS path BGP AS path

42. Sibling ASes • Single organization owns and manages multiple ASes • May share address space • Large fan-in and fan-out for the “sibling AS pair”

43. Sibling ASes cause extra AS hop • Large fan-in and fan-out for the “sibling AS pair” E E A A F F B B H D D G G C C Traceroute AS path BGP AS path

44. Unannounced infrastructure addresses • ASes do not necessarily announce infrastructure via BGP • Lead to “unmapped” addresses • Sometimes fall into supernet announced by AS’s provider or sibling

45. 3. B,A 4. A,C,A 2. A 1. A,C Unannounced infrastructure addresses AS loop in traceroute path AS A Substitute AS hop AS B Missing AS hop in traceroute path AS C Extra AS hop in traceroute path

46. BGP path and traceroute path could differ! • Inaccurate IP-to-AS mapping • Traceroute problems • Forwarding path changing during traceroute • Interface numbering at AS boundaries • ICMP response refers to outgoing interface • Legitimate mismatches

47. Forwarding path changing during traceroute AS D AS E Route flaps between A B C and A D E AS A AS B AS C AS A AS D AS C AS hop B is substituted by AS D in the traceroute path

48. Interface numbering at AS boundaries AS A AS C AS A AS B AS C Missing AS hop B in traceroute path

49. ICMP response refers to outgoing interface AS A AS C ICMP message AS B Extra AS hop B in traceroute path

50. BGP path and traceroute path could differ! • Inaccurate IP-to-AS mapping • Traceroute problems • Legitimate mismatches • Route aggregation and filtering • Routing anomalies, e.g., deflections