Internet Routing (COS 598A) Today: Interdomain Topology

1. Internet Routing (COS 598A)Today: Interdomain Topology Jennifer Rexford http://www.cs.princeton.edu/~jrex/teaching/spring2005 Tuesdays/Thursdays 11:00am-12:20pm

2. Outline • Interdomain topology • AS graph • Inferring the topology from routing data • Structure of the AS graph • AS relationships • Common pair-wise relationships • Inferring the relationships from routing data • Characteristics of the relationship graph • Peering policies • Example of AOL’s peering agreement

3. 4 3 5 2 6 7 1 What is the AS Graph? • Node: Autonomous System • Edge: Two ASes that speak BGP to each other

4. What is an Edge, Really? • Edge in the AS graph • At least one BGP session between two ASes • Some destinations reached from one AS via the other d d AS 1 AS 1 Exchange Point AS 2 AS 3 AS 2

5. How Do You Know a Node or Edge Exists? • Consult the Whois database? • Tells which ASes have been allocated • But, might be out-of-date on who owns it • … and often doesn’t say who the neighbors are • See a path that uses the node/edge • Collect measurements of AS paths • Extract all of the nodes and edges • E.g., AS path “7018 1 88” implies • Nodes: 7018, 1, and 88 • Edges: (7018, 1) and (1, 88)

6. Interdomain Routing: Border Gateway Protocol • ASes exchange info about who they can reach • IP prefix: block of destination IP addresses • AS path: sequence of ASes along the path • Example: a BGP route in AS 7018 shows • Prefix 12.34.158.0/24 has path “7018, 1, 88” “12.34.158.0/24: path (88)” “12.34.158.0/24: path (7018,1,88)” 88 1 7018 data traffic data traffic 12.34.158.5

7. Where to Get BGP Routes: Public Servers 4 7018 1221 701 3786 7 80 9.184.112.0/20 3.0.0.0/8 BGP sessions

8. Example: BGP Table (“show ip bgp” at RouteViews) Network Next Hop Metric LocPrf Weight Path * 3.0.0.0/8 205.215.45.50 0 4006 701 80 i * 167.142.3.6 0 5056 701 80 i * 157.22.9.7 0 715 1 701 80 i * 195.219.96.239 0 8297 6453 701 80 i * 195.211.29.254 0 5409 6667 6427 3356 701 80 i *>12.127.0.249 0 7018 701 80 i * 213.200.87.254 0 3257 701 80 i * 9.184.112.0/20 205.215.45.50 0 4006 6461 3786 i * 195.66.225.254 0 5459 6461 3786 i *>203.62.248.4 0 1221 3786 i * 167.142.3.6 0 5056 6461 6461 3786 i * 195.219.96.239 0 8297 6461 3786 i * 195.211.29.254 0 5409 6461 3786 i AS 80 is General Electric, AS 701 is UUNET, AS 7018 is AT&T AS 3786 is DACOM (Korea), AS 1221 is Telstra

9. All ASes have degree >= 1 Very few have degree >= 100 Characteristics of the AS Graph • AS graph structure • High variability in node degree (“power law”) • A few very highly-connected ASes • Many ASes have only a few connections 1 0.1 CCDF 0.01 0.001 AS degree 1 10 100 1000

10. Characteristics of AS Paths • AS path may be longer than shortest AS path • Router path may be longer than shortest path 2 AS hops, 8 router hops d s 3 AS hops, 7 router hops

11. d2 d1 Problem of Missing Edges • Limited collection of paths • Some edges might never be traversed • Especially low in the AS hierarchy • … and backup links • Example: paths from two tier-1 ISPs miss an edge Sprint AT&T ??? Harvard B-school Harvard

12. Problem of Missing Nodes • Route aggregation • AS advertises a larger address block • Smaller address block not seen everywhere • Can cause an AS not to appear in BGP table • C’s table: has paths “C”, “C D”, and “C E” • B’s table: has only path “C” for 12.0.0.0/8 C: 12.0.0.0/8 D: 12.1.0.0/16 D A B C E E: 12.2.0.0/16

13. Research Questions • Incomplete data • How to get more data? • How much does missing data affect answers? • What kinds of questions can be answered safely?

14. AS Relationships

15. “12.34.158.0/24: path (2,1)” “12.34.158.0/24: path (1)” 2 3 Interdomain Routing Policies • Two main decisions • Path selection: which of the paths to use? • Path export: which neighbors to tell? • Both driven by business relationships, e.g., • Customer pays provider for Internet access • Peers find it mutually advantageous to cooperate 1 data traffic data traffic 12.34.158.5

16. advertisements traffic Customer-Provider Relationship • Customer needs to be reachable from everyone • Provider exports routes learned from customer to everyone • Customer does not want to provide transit service • Customer does not export from one provider to another Traffic to the customer Traffic from the customer d provider provider customer d customer

17. advertisements traffic Peer-Peer Relationship • Peers exchange traffic between customers • AS exports only customer routes to a peer • AS exports a peer’s routes only to its customers Traffic to/from the peer and its customers peer peer d

18. two peer edges transit through a customer Paths You Should Never See (“Invalid”) Customer-provider Peer-peer

19. Other Kinds of Relationships • Siblings • Same company • Mutual transit service • Like one bigger AS • Mergers, acquisitions, … • Backup • Used only when failure • Second provider • Backup peering • Geography-specific • Customer in U.S. • Peer in Europe E A F B H D G C primary backup

20. AS Relationships Matter • Scientific understanding • Understanding Internet structure and evolution • Understanding why certain paths are used for traffic • Placement of Web servers • Want to be close to most customer networks • Business decisions • Selecting new peer or provider, or renegotiating relations • Security policies • Knowing which BGP routes look suspicious • Analyzing BGP convergence • Relationships have a big impact here (more later!)

21. Inferring AS Relationships • Top down: how routes are selected • AS relationships define routing policy • Routing policy determines the routes you see • Bottom up: how policies can be inferred • Routing data are available from public sources • The chosen routes tell you about the policy • Example: seeing path “A B C” tells you… • B permits A to transit through B to reach C • (A,B) and (B,C) cannot both be peering links • A and C are not both upstream providers of B

22. Type-of-Relationship Problem • Given the inputs • AS graph G(V,E) with vertices V and edges E • Set of paths P on the graph G • Find a solution that • Labels each edge with an AS relationship • Minimizes the number of “invalid” paths in P • Rich area of research work • http://www-unix.ecs.umass.edu/~lgao/ton.ps • http://www.cs.princeton.edu/~jrex/papers/infocom02.pdf • http://www.cs.berkeley.edu/~sagarwal/research/BGP-hierarchy/ • … lots of scope for algorithms-oriented research project

23. AS Relationship Graph (2002) • Lowest level: Stubs • Leaf nodes: no peers or downstream customers • 8898 of the 10915 ASes (82.5% of ASes) • Ex: UC Berkeley (25), Princeton (88), AT&T Labs (6431), and INRIA (1300) • Next lowest level: Regional ISPs • Leaf nodes after successive pruning of leaf nodes • 971 ASes of the 10915 ASes (8.9% of ASes) • Ex: PacBell (5676), US West (6223), and UUNET Canada (815) • Remaining 1046 ASes: Core

24. AS Relationship Graph (2002), Continued • Dense core: Tier-1 providers • (Nearly) fully-connected nodes with no providers • Around 15-20 ASes in a near-clique • Ex: Sprint, UUNET, AT&T, Verio, Level3, C&W,… • Transit core: • ASes that peer with the dense core and each other • 129 ASes, including top providers in Europe and Asia • Ex: UUNET Europe, KDDI, and Singapore Telecom • Outer core • All of the remaining ASes in the core • 897 ASes, including large regional and national ISPs • Ex: Turkish Telecom and Minnesota Regional Network

25. Node Degree is Not Enough • Node degree ignores relationships • A stub AS may have many upstream providers • A core AS may have a small number of peers • Some ASes have customers that don’t have AS numbers

26. Ideas for Class Projects • AS relationship inference • New algorithms for inferring AS relationships • Longitudinal study of AS relationship graph • Influence of policies on measuring AS graph • AS peering policies • Analysis of incentives to peer or not • Study of how one AS can game another • Analysis of whether regulation is necessary to keep large ASes from locking out smaller ones • Alternate settlement models • Associating prices with routes? • Source-based routing with third-party control?

27. Peering Policies • Contracts that outline: • Operational requirements on peer network • Backbone and peering capacity requirements • Number and geographic diversity of peering points • Volume and ratio of traffic between two peers • Routing-policy requirements • AOL’s Settlement-Free Interconnection Policy • http://www.atdn.net/settlement_free_int.shtml

28. AOL’s Settlement-Free Interconnection Policy • Operational requirements on a peer network • Handle a single-node outage w/o traffic impact • Single AS number • Network Operations Center staffed at all times • Backbone capacity • At least 10 gigabits/sec between 8 or more cities • Minimum peering link speed of 622 megabits/sec • Peering locations (in U.S.) • At least four locations • Must include D.C. area, middle of country, Bay area, and NYC or Atlanta

29. Efficient Early-Exit Routing • Diverse peering locations • Both costs, and middle • Comparable capacity at all peering points • Can handle even load • Consistent routes • Same destinations advertised at all points • Same AS path length for a destination at all points Customer B Provider B multiple peering points Early-exit routing Provider A Customer A

30. AOL Routing Requirements • Consistent advertisements • All customer routes • At all peering points • With the same AS path length • Address blocks • Routes aggregated as much as possible • No address blocks smaller than /24 • Address blocks are registered (e.g., with ARIN) • No default routing • Only send traffic to destinations AOL advertises

31. For Next Tuesday’s Class • Adapting routing inside an AS to the traffic • “The Revised ARPANET Routing Metric” (1989) • “Traffic Engineering With Traditional IP Routing Protocols” (survey paper, 2002) • Written one-page review of first paper • Brief summary of the paper • Reasons to accept the paper • Reasons to reject the paper • Suggestions for future research directions • Just to skim… • RFC 3272: “Overview and Principles of Internet Traffic Engineering”