Can the Border Gateway Protocol (BGP) be fixed?

1. Can the Border Gateway Protocol (BGP) be fixed? Timothy G. Griffin Intel Research, Cambridge UK tim.griffin@intel.com UCL Oct 15, 2003

2. How do you connect to the Internet? Physical connectivity is just the beginning of the story….

3. Architecture of Dynamic Routing IGP EGP (= BGP) AS 1 IGP IGP = Interior Gateway Protocol Metric based: OSPF, IS-IS, RIP, EIGRP (cisco) AS 2 EGP = Exterior Gateway Protocol Policy based: BGP The Routing Domain of BGP is the entire Internet

4. BGP Table Growth Thanks to Geoff Huston. http://bgp.potaroo.net on May 30, 2003

5. How Many ASNs are there? Thanks to Geoff Huston. http://bgp.potaroo.net on May 30, 2003

6. Partial View of www.cam.ac.uk (131.111.8.46) Neighborhood AS 20757 Hanse AS 5089 NTL Group AS 3356 Level 3 AS 3257 Tiscali AS 6461 AboveNet AS 1239 Sprint AS 702 UUNET AS 13127 Versatel AS 4637 REACH AS 20965 GEANT AS 786 ja.net (UKERNA) AS 5459 LINX AS 1213 HEAnet (Irish academic and research) Originates > 180 prefixes, Including 131.111.0.0/16 AS 4373 Online Computer Library Center AS 7 UK Defense Research Agency

7. Topology information is flooded within the routing domain Best end-to-end paths are computed locally at each router. Best end-to-end paths determine next-hops. Based on minimizing some notion of distance Works only if policy is shared and uniform Examples: OSPF, IS-IS Each router knows little about network topology Only best next-hops are chosen by each router for each destination network. Best end-to-end paths result from composition of all next-hop choices Does not require any notion of distance Does not require uniform policies at all routers Examples: RIP, BGP Technology of Distributed Routing Link State Vectoring

8. BGP Route Processing Open ended programming. Constrained only by vendor configuration language Apply Policy = filter routes & tweak attributes Apply Policy = filter routes & tweak attributes Receive BGP Updates Based on Attribute Values Best Routes Transmit BGP Updates Apply Import Policies Best Route Selection Best Route Table Apply Export Policies Install forwarding Entries for best Routes. IP Forwarding Table

9. Shedding Inbound Traffic with ASPATH Prepending Prepending will (usually) force inbound traffic from AS 1 to take primary link AS 1 provider 192.0.2.0/24 ASPATH = 2 2 2 192.0.2.0/24 ASPATH = 2 primary backup customer Yes, this is a Glorious Hack … 192.0.2.0/24 AS 2

10. … But Padding Does Not Always Work AS 1 AS 3 provider provider 192.0.2.0/24 ASPATH = 2 192.0.2.0/24 ASPATH = 2 2 2 2 2 2 2 2 2 2 2 2 2 2 AS 3 will send traffic on “backup” link because it prefers customer routes and local preference is considered before ASPATH length! Padding in this way is often used as a form of load balancing primary backup customer 192.0.2.0/24 AS 2

11. COMMUNITY Attribute to the Rescue! AS 3: normal customer local pref is 100, peer local pref is 90 AS 1 AS 3 provider provider 192.0.2.0/24 ASPATH = 2 COMMUNITY = 3:70 192.0.2.0/24 ASPATH = 2 primary backup Customer import policy at AS 3: If 3:90 in COMMUNITY then set local preference to 90 If 3:80 in COMMUNITY then set local preference to 80 If 3:70 in COMMUNITY then set local preference to 70 customer 192.0.2.0/24 AS 2

12. Don’t celebrate just yet… Provider A (Tier 1) Provider B (Tier 1) peering provider/customer provider/customer Provider C (Tier 2) customer Now, customer wants a backup link to C….

13. Customer installs a “backup link” … Provider A (Tier 1) Provider B (Tier 1) Provider C (Tier 2) primary backup customer sends “lower my preference” Community value customer

14. Disaster Strikes! Provider A (Tier 1) Provider B (Tier 1) Provider C (Tier 2) primary backup customer customer is happy that backup was installed …

15. The primary link is repaired, and something odd occurs… Provider A (Tier 1) Provider B (Tier 1) Provider C (Tier 2) primary backup customer YIKES --- routing DOES NOT return to normal!!!

16. WAIT! It Gets Better… A B B B P C B D P = primary B = backup

17. OOOOOPS! A B B B P C B No solution = Protocol Divergence D Suppose A, B, C all break ties in the same direction (clockwise or counter-clockwise)

18. What the heck is going on? • There is no guarantee that a BGP configuration has a unique routing solution. • When multiple solutions exist, the (unpredictable) order of updates will determine which one is wins. • There is no guarantee that a BGP configuration has any solution! • And checking configurations NP-Complete [GW1999] • Complex policies (weights, communities setting preferences, and so on) increase chances of routing anomalies. • … yet this is the current trend!

19. Larry Speaks Is this any way to run an Internet? http://www.larrysface.com/

20. What Problem is BGP Solving? Underlying problem Distributed means of computing a solution. Shortest Paths RIP, OSPF, IS-IS ???? Stable Paths BGP [GSW1998, GSW2002]

21. Separate dynamic and static semantics “static” semantics dynamic semantics Booo Hooo, Many, many complications... BGP BGP Policies SPVP = Simple Path Vector Protocol, a distributed algorithm for solving SPP Stable Paths Problem (SPP) SPVP Worst case, This is an exponential Time and space translation

22. An instance of the Stable Paths Problem (SPP) 2 1 0 2 0 5 2 1 0 4 2 0 4 3 0 1 4 2 0 5 3 3 0 1 3 0 1 0 2 • A graph of nodes and edges, • Node 0, called the origin, • For each non-zero node, a set or permitted paths to the origin. This set always contains the “null path”. • A ranking of permitted paths at each node. Null path is always least preferred. (Not shown in diagram) 1 most preferred … least preferred When modeling BGP : nodes represent BGP speaking routers, and 0 represents a node originating some address block

23. A Solution to a Stable Paths Problem 5 2 1 0 5 1 0 2 4 3 2 2 1 0 2 0 A solution is an assignment of permitted paths to each node such that 4 2 0 4 3 0 • node u’s assigned path is either the null path or is a path uwP, where wP is assigned to node w and {u,w} is an edge in the graph, • each node is assigned the highest ranked path among those consistent with the paths assigned to its neighbors. 3 0 1 3 0 1 0 1 A Solution need not represent a shortest path tree, or a spanning tree.

24. An SPP may have multiple solutions 1 1 1 0 0 0 2 2 2 1 2 0 1 0 1 2 0 1 0 1 2 0 1 0 2 1 0 2 0 2 1 0 2 0 2 1 0 2 0 First solution Second solution DISAGREE

25. BAD GADGET : No Solution 2 1 0 2 0 2 4 0 3 2 0 3 0 1 3 0 1 0 3 3 1 This is an SPP version of the example first presented in Persistent Route Oscillations in Inter-Domain Routing. Kannan Varadhan, Ramesh Govindan, and Deborah Estrin. Computer Networks, Jan. 2000

26. SURPRISE! 2 1 0 2 0 Becomes a BAD GADGET if link (4, 0) goes down. 2 4 0 4 2 0 4 3 0 BGP is not robust : it is not guaranteed to recover from network failures. 4 0 3 1 3 4 2 0 3 0 1 3 0 1 0

27. Can BGP be fixed? • BGP policy languages have evolved organically • A policy language really should be designed! • But how? Joint work with Aaron Jaggard (UPenn Math) and Vijay Ramachandran (Yale CS) SIGCOMM 2003

28. Design Dimensions • Robustness (required!) • Transparency (required!) • Expressive Power • Autonomy (“freedom of independent action”) • Global Consitency • Policy Opaqueness Tradeoffs abound

29. Robustness Partially Partially Ordered (PP0): For all paths P and Q, (P < Q and Q < P) implies (P = Q or last(P) = last(Q)) P < Q : transitive closure of (subpath relation on permitted paths union the path ranking relation at each node) This is a sufficient condition for robustness Checking robustness is an NP-hard

30. Transparency, Autonomy • Transparency: protocol will compose its transformation with transformation of policy writer. • Autonomy: measure of “wiggle room” • Weak autonomy: neighbors can’t dictate relative ranking of routes • Stronger: policy writer can classify neighbors and rank routes based on class (“autonomy of neighbor ranking”).

31. Need Global Constraints Theorem: Any robust system supporting both transparency and autonomy must have a non-trivial global constraint Global constraints must be a part of design from the start

32. A Partial Ordered for the Design Space ( J , L ) < ( J , L ) 1 1 2 2 Global Constraint Local Constraint if and only if for all S : SPP • J(S)impliesJ(S) • L(S)impliesL(S) 2 1 1 2 2

33. Robust Designs ( J, L ) is robust if and only if (J andL ) impliesPPO Examples: ( True, SP ) 2 ( PPO, True )

34. Robust Subspace Not tractable Tractable ( PPO, True ) Constraint Simplicity Expressive Power ( True, SP )

35. Hierarchical BGP (HBGP) HBGP +PEER + BU HBGP + BU HBGP +PEER HBGP [GR2000, GGR2001]

36. Next? • Need techniques for constructing policy languages. • Design of protocols to enforce global constraints. • Is there a general formalism to capture autonomy?

37. References • [VGE1996, VGE2000] Persistent Route Oscillations in Inter-Domain Routing. Kannan Varadhan, Ramesh Govindan, and Deborah Estrin. Computer Networks, Jan. 2000. (Also USC Tech Report, Feb. 1996) • [GW1999] An Analysis of BGP Convergence Properties. Timothy G. Griffin, Gordon Wilfong. SIGCOMM 1999 • [GSW1999] Policy Disputes in Path Vector Protocols. Timothy G. Griffin, F. Bruce Shepherd, Gordon Wilfong. ICNP 1999 • [GW2001] A Safe Path Vector Protocol. Timothy G. Griffin, Gordon Wilfong. INFOCOM 2001 • [GR2000] Stable Internet Routing without Global Coordination. Lixin Gao, Jennifer Rexford. SIGMETRICS 2000 • [GGR2001] Inherently safe backup routing with BGP. Lixin Gao, Timothy G. Griffin, Jennifer Rexford. INFOCOM 2001 • [GW2002a] On the Correctness of IBGP Configurations. Griffin and Wilfong.SIGCOMM 2002. • [GW2002b] An Analysis of the MED oscillation Problem. Griffin and Wilfong. ICNP 2002.