Consensus Routing

1. Consensus Routing Antonio-Gabriel Sturzu, SCPD

2. Table of Contents • Introduction • Consistency issues • Consensus Routing Overview • Stable Mode • Transient Mode • Performance and Overhead

3. Introduction • Internet routing, especially interdomain routing has favored responsiveness over consistency • In interdomain routing a router applies a received update immediately to its forwarding table before propagating it to other routers • BGP updates are known to cause up to 30% packet loss for two minutes or more after a routing change • Transient loops account for 90% of all packet loss

4. Introduction(2) • The primary contribution of the article is that is that it separates the safety concept from the liveness concept and associates consistency with safety and responsiveness with liveness • Consistency safety means that a router forwards a packet along a packet adopted by the upstream routers • Liveness means that the system reacts quickly to failures or policy changes • Separating safety and liveness improves end-to-end availability • They are obtained through stable and transient modes

5. Consistency Issues • BGP link failures

6. Consistency issues(2) • BGP policy change

7. Consistency issues(3) • iBGP link recovery • Such blackholes can cause packet loss for tens of seconds

8. Consistency issues(4) • BGP policy cycles

9. Consensus Routing Overview • Forwards packets using • Stable mode • Transient mode • Consensus routers simply log the new routes computed by the policy engine • Periodically all routers engage in a distributed coordination algorithm that determines the most recent set of complete updates

10. Consensus Routing Overview(2) • The coordination is based on classical distributed snapshot and consensus algorithms • The routers use the output of the coordination to compute a set of stable forwarding tables (STFs) that are guaranteed to be consistent

11. Stable Mode • The distributed coordination algorithm proceeds in epochs • Steps of an epoch k: • Update log • Distributed snapshot • The snapshot is a globally consistent view of all the updates in the system (complete or incomplete) • Frontier computation • Aggregation • Consensus • Flood

12. Stable Mode(2) • SFT computation • View change • Versioning • Garbage colection

13. Router State • Routing Information Base (RIB) • Stores for each destination • Route update received from each neighbor • Locally selected best route • Route advertised to each neighbor • History • Stores for each destination a chronological list of received and selected routes in the RIB • SFTs • Store for each destination the next-hop interfaces corresponding to the stable routes

14. Router State(2) • Triggers • Globally unique identifier for a set of causally related events propagating through the network • (AS number, trigger number) • In consensus routing each update carries a trigger that is associated with the route being implicitly withdrawn and replaced by the route announced in the update • It tracks when the implicit withdrawal is complete

15. Router State(3) • In order to maintain the safety property an AS A generates a new trigger to be sent along with an update upon • A failure of the next-hop in A’s current route to the destination • A policy change that causes A to prefer another route to the destination over the current one • Receiving a route from a neighbor B that it prefers over its current route via a different neighbor C

16. Update Processing

17. Distributed Snapshot

18. Frontier Computation • Aggregation • Send the set of triggers (complete or incomplete) • Consensus • Consolidators ensure that • There is no single point of failure • No single AS is trusted with the task of consolidating the snapshot • A consolidator is reachable from every AS with high probability • When consensus ends the consolidators use the snapshot report in order to compute the set of incomplete triggers I in the network

19. Frontier Computation(2) • In order to compute the set I they use the following idea: • A trigger is said to depend on all trigers that precede it in the history table • A trigger t is said to be complete if neither t nor any of his predecessors are incomplete • Flood • The set of incomplete triggers I and the set S of AS-es that succesfully participated in the distributed snapshot are sent to all AS-es

20. Building SFTs

21. Transient Mode • Routing deflections • Backtracking • Detour routing • Backup routes • Use RBGP • Choosing the most link-disjoint backup route from the primary route protects against single link failures

22. Performance • Link failures • For BGP 13% of failures cause at least half of all AS-es to experience routing loops • For Consensus Routing with transient forwarding • Backtraking enables continuous connectivity for at least 74% of all AS-es following 99% of failure cases • By detouring connectivity is 98.5% • With backup routes connectivity is 98%

23. Performance • Policy change • For BGP in more than 55% of the test cases AS-es were disconnected from the destination due to transient loops formed during convergence • Consensus routing transitions from one set of consistent loop-free routes to another completely avoiding transient loops

24. Overhead • Volume of control traffic

25. Overhead(2) • Cost of consensus • For 9 nodes all the nodes learnt the agreed value in under 450 miliseconds • For 18 and 27 nodes times were 1.4 and 1.8 seconds • Path dilation • Measures how far packets have to be redirected

26. Overhead(3) • Path dilation

27. Overhead(4) • Response time • A 30 second epoch results in more than 90% of the paths being adopted in less than 2 minutes

28. Overhead(5) • Implementation Overhead • Consensus Routing adds 8% in update processing and about 11% additional lines of code to the BGP implementation