MIRO : M ulti-path I nterdomain RO uting

1. Wen XuJuly 30, 2009 MIRO:Multi-pathInterdomainROuting

2. Internet Routing : current status AS 1 AS7018 AS 701 AS 29 AS 88 AS = Autonomous System31,311 ASes in 2009 Internet The Internet

3. Interdomain Routing Interdomain routing is important 31,311 ASes in April 2009 Most traffic traverses multiple ASes Interdomain routing is challenging A federation of multiple independent ASes Economic factors drive routing policies Not necessarily using shortest path People are not willing to reveal internals

4. Our Contribution: MIRO Motivation Current protocol lacks flexibility and control MIRO: a new Interdomain routing protocol Offers substantial flexibility Avoid state explosion in disseminating info Give ASes control over the traffic in their networks Backward compatible and incrementally deployable

5. Outline of The Talk Motivation and Related Work Design and Implementation Measurement Based Evaluation Guarantee the Convergence

6. Current Interdomain Routing Protocol: BGP (Border Gateway Protocol)‏ Path-Vector Protocol:Each node calculates its AS paths to each reachable destination and advertises that to adjacent neighbors Only a single route is chosen CF* CF*CEFCBEF CF*CEF BEF*BCF ABEF*ADEF DEF* EF *ECF EF * B C F* A F D E DEF*DABEF

7. When BGP is not enough: Avoiding an AS BEF*BCF CF*CEFCBEF • AS A does not trust AS E to carry its traffic • AS A wants the route BCF in AS B instead of BEF • Even if B is willing to satisfy A’s request, B can not do that because all of B’s neighbors would also switch to BCF • BCF is available, it is just not advertised or used ABEF*ADEF B C F* A F D E DEF*DABEF EF *ECF

8. When BGP is not enough: Incoming Traffic Control Too much traffic uses EF, CF barely used Receiving AS has little say over which path to use People do want that in today’s Internet 12,468 stubs out of 31,311 ASes multi-homed More than 4,900 ASes use AS prepending BEF*BCF CF*CEFCBEF ABEF*ADEF B C F* A F D E DEF*DABEF EF *ECF

9. The High Level Problem Flexibility: single path routing limits flexibility Different people have different needs They care about properties of the paths Efficiency: avoid disclosing states unnecessarily Most ASes are satisfied with BGP routes Control: give intermediate ASes more control Get more alternative paths Current business relationship limited to adjacent ASes

10. Option 1: BGP Flexibility No, single path limits flexibility Efficiency Yes Control Yes, but not perfect ASes use local policies to control its traffic But very hard to control non-adjacent ASes

11. Option 2: Source Routing Flexibility Yes, ultimate flexibility Efficiency No, all internal states exposed Control No, intermediate ASes lose control B ABCF C ABEF A F D ADECF E

12. At Another Level: Overlays Virtual networks above physical networks Additional cost in setting up overlay boxes They have no control over physical paths B C A F D Overlay nodes E

13. Our Proposal: MIRO Flexibility Expose the alternative routes already there Efficiency Use BGP for default routes Explore alternatives only when necessary Control Intermediate ASes still have routing policies They can negotiate with an arbitrary AS They can control what routes they expose

14. Outline of The Talk Motivation and Related Work Design and Implementation Measurement Based Evaluation Guarantee the Convergence

15. MIRO Design: AS-level Path-Vector Protocol AS-level routing How do you divide the boundaries AS as a natural entity of trust and policy specifications Each AS decides how traffic should flow Path-vector protocol Path-vector protocol fits “federated” world Build AS paths along the way Downstream AS should be able to control who may send traffic through its network

16. MIRO Design: Route Negotiation Pull-based route retrieval Solicit routes only when necessary Bilateral negotiations Business relationships usually bilateral anyway Any route to F avoiding E? BCF is OK BCF BEF*BCF CF*CEFCBEF B C ABEF*ADEF A F F* D E DEF*DABEF EF *ECF

17. MIRO Design: Negotiation with Anyone Negotiate with adjacent/non-adjacent ASes Negotiate with ASes in either direction Sender-side negotiation Receiver-side negotiation BEFBCF* ABCF*ADEF BCF OK Yes BCF Any route to F avoiding EF? DEF*DABCF BEF*BCF CF*CEFCBEF ABEF*ADEF B C A F F* D E DEF*DABEF EF *ECF

18. MIRO Design: Routing Tunnels Destination based routing is no longer sufficient Tunnels transmit traffic between two endpoints Normally implemented by IP-in-IP encapsulation Assign unique tunnel id upon successful negotiation New IP header New IP header Old packet Old packet Old packet B C A F D E

19. MIRO Implementation: Revisiting Intra-AS Structure AS F AS C AS E R2(12.34.56.2)‏ R3(12.34.56.3)‏ AS B(12.34.56.0/24)‏ Egress routers Use IBGP todistribute routes R1(12.34.56.1)‏ Ingress routers AS A

20. What IP Do We Use in Encapsulation? IP of exit links IP of egress routers One IP for all tunnels Rewrite to IP of egress router at ingress routers AS C AS E R2(12.34.56.2)‏ R3(12.34.56.3)‏ AS B(12.34.56.0/24)‏ Egress routers R1(12.34.56.1)‏ Ingress routers AS A

21. When to Destroy Tunnels Terminating the business contract Either party may tear it down actively BGP route is withdrawn or changed The route from upstream to downstream The route selected at the responding AS Failure of the ASes

22. Outline of The Talk Motivation and Related Work Design and Implementation Measurement Based Evaluation Guarantee the Convergence

23. Evaluation Outline Flexibility MIRO does provide more flexibility Efficiency Not exploring too much state in negotiations Control ASes can choose strict or flexible policy, strict policy already gives much benefit

24. Evaluation Methodologies How do we evaluate a new routing protocol without deploying it Something resembling the Internet The behavior of routing protocols governed by local policies Where do we find local policies

25. Local Policies in Today’s Internet sibling sibling UUNET AT&T 2 peer peer $$ Export rules: Advertise all paths to customers or siblings Advertise only customer routes to peers or providersPreference rules: Prefer customer routes over peer or provider routes AT&T provider customer Princeton

26. Evaluation Methodologies Local policies not available Use business relationships to approximate Business relationships not available Use relationship inferring algorithm which takes real BGP dumps as input They are not perfect, but they are the closest we can get toward Internet

27. The Data 2000/2003/2005 RouteView BGP dump Gao’s AS relationship inference algorithm Each AS is one node in the topology Date AS # Link # P/C links Peering links Sibling Links 10/1/2000 8829 17793 16531 1031 231 10/8/2003 16130 34231 30649 3062 520 10/8/2005 20930 44998 40558 3753 687

28. Node Degree Distribution 1/3 of nodes with only 1 edge 10/48/80 nodes with degree > 200 40% of nodes with 2 edges

29. Evaluation Methodologies Evaluated applications Avoid an AS for security or performance Controlling incoming traffic Evaluated policies Strict policy (/s)‏ Respect export policy (/e)‏ Most flexible policy (/a)‏ Only build one tunnel Negotiate with adjacent neighbors and the ASes on the default BGP path only (avoiding AS)‏

30. Avoiding AS: success rate BGP only gets around 30% Strict policy can already get around 65% More flexible policy can get around 76% Even source routing can only push 10% more Date BGP MIRO/s MIRO/e MIRO/a SourceRouting 2000 27.8% 65.4% 72.9% 75.3% 89.5% 2003 31.2% 67.0% 74.6% 76.6% 90.4% 2005 29.5% 67.8% 73.7% 76.0% 91.1%

31. Avoiding AS: state explosions With more flexible policies Number of ASes we negotiated with decrease Number of paths increase Results on 2000 and 2003 data Number of ASes roughly the same Number of paths increase with time Policy Success rate AS #/tuple Path #/tuple 2005 Strict 67.8% 2.80 36.6 Export 73.7% 2.53 58.9 Flexible 76.0% 2.38 139.0

32. Avoiding AS: Incremental Deployment 99.9% of total gain if 25% of nodes adopted MIRO 53% of total gain if 0.2% of nodes (40 nodes) adopted MIRO 82% of total gain if 25% of nodes adopted MIRO 44% of total gain if 0.2% of nodes (40 nodes) adopted MIRO

33. Another Real Application:Incoming Traffic Control Assume that a multi-homed stub AS wants to balance incoming traffic How much traffic can it move by just negotiating with one upstream AS 10,383 stubs out of 20,930 ASes (2005 data)‏ 90% can move more than 10% of the traffic 50% can move more than 40% (flexible policy)‏ 50% can move more than 25% (strict policy)‏

34. Another Real Application:Incoming Traffic Control (cont)‏ In the MIRO paper, we assume stub AS blindly obey selection made by power node, here, stub AS re-run BGP selection process 10,383 stubs out of 20,930 ASes (2005 data)‏ 77% can move more than 10% of the traffic 28% can move more than 40% (flexible policy)‏ 50% can move more than 18% (strict policy)‏

35. Outline of The Talk Motivation and Related Work Design and Implementation Measurement Based Evaluation Guarantee the Convergence

36. The Convergence Problem Oscillation is bad, it leads to lost packets BGP does not guarantee convergence MIRO introduces more flexibility, so it opens up new problems for convergence Certain policies guarantees convergence One tunnel + do not advertise to others Advertise to stub ASes only

37. Convergence Problem:BGP May Not Converge A counter-example that does not converge AX ABX AX AX ABX CAX BX BCX CX BCX BX BX CAX CX CX A A prefers ABX over AXB prefers BCX over BXC prefers CAX over CX X B C

38. Topology + Policy -> Convergence Many scenarios do not happen in the real world A few prevalent relationships Provider/Customer, Peer/Peer, Sibling/Sibling Topology Provider/customer relationships implies a partial order Policy: Export policy -> some paths never occur Preference policy -> some paths always favored Conclusion:BGP converges without global coordination

39. The Convergence of MIRO Counter-examples show that MIRO does not converge without restrictions Proved that MIRO would converge if certain policy guidelines are obeyed Guideline A is the original Guideline that guarantees the convergence of BGP:customer routes are always preferred over any provider routes or peer routes

40. Guideline B: Tunnels as a Higher Level Tunnels constructed using only BGP and not advertised back to BGP BCEFBEF*BCF CF*CEFCBEF B C ABCFABEF*ADEF A F D E

41. Guideline C: Advertise Tunnels to Stub ASes Only Advertise tunnels as BGP paths to stubs only BCEFBEF*BCF CF*CEFCBEF B C ABCEFABEF*ADEF A F D E Stub node

42. Strict Policy Does Not Guarantee Convergence

43. Strict Policy with Restriction Guarantees Convergence Guideline D:Require that each AS enforces a strict order such that first_downstream(r) ≺ a(r.prefix)‏ Guideline E:Only allow a tunnel if the route from current node to first_downstream(r) does not contain a tunnel

44. Mixing and Matching the Guidelines Guideline A can be replaced with the other Guidelines in the original paper Each AS can choose between A+C and A+D, or they can choose between A+C and A+E Guideline B-tunnel can be built on top of Guideline C-tunnel, Guideline D-tunnel, or Guideline E-tunnel

45. These Guidelines Are Practical Most end-to-end paths contain only 1 tunnel Many ASes are stub ASes Average AS path is 4 Negotiations allowed between non-adjacent ASes Most stub ASes can be satisfied by Guideline C Economic incentives motivate for strict policy, and the restrictions are easy to implement

46. Summary MIRO: Multi-path Interdomain ROuting Use BGP for default routes, negotiate only when necessary Give more power to individual ASes Incrementally deployable Much flexibility achieved using one tunnel Can adopt flexible policies

47. Thanks