Internet Routing (COS 598A) Today: Multi-Homing

1. Internet Routing (COS 598A)Today: Multi-Homing Jennifer Rexford http://www.cs.princeton.edu/~jrex/teaching/spring2005 Tuesdays/Thursdays 11:00am-12:20pm

2. Outline • Multi-homing • Motivations: reliability, performance, and financial • Do you really need to use a routing protocol? • Controlling outbound traffic • Shortest-path routing • Primary and backup providers • Load balancing over multiple links • Controlling inbound traffic • Primary and backup providers • Selective advertising • BGP communities • State-of-the-art today

3. Why Connect to Multiple Providers? • Reliability • Reduced fate sharing • Survive ISP failure • Performance • Multiple paths • Select the best • Financial • Leverage through competition • Game 95th-percentile billing model Provider 2 Provider 1

4. The Stub AS Doesn’t Need to Speak BGP… • Sending traffic • Assume both providers can reach everyone • Split traffic however you want (e.g., 50%/50%) • But… what if a provider can’t reach someone? • But… what if one provider has a better path? Provider 1 Provider 2 One static route L1 L2 0.0.0.0/0  L1, L2

5. The Stub AS Doesn’t Need to Speak BGP… • Receiving traffic • Both providers can announce the prefix into BGP • Ensures that everyone else can reach you • But… what if traffic load is very uneven? Advertise 12.34.158.0/24 Provider 1 Provider 2 traffic traffic 12.34.158.0/24

6. Controlling Outbound Traffic

7. Outbound Traffic: Pick a BGP Route • Easier to control than inbound traffic • IP routing is destination based • Sender determines where the packets go • Control only by selecting the next hop • Border router can pick the next-hop AS • Cannot control selection of the entire path Provider 1 Provider 2 “(1, 3, 4)” “(2, 7, 8, 4)”

8. Outbound Traffic: Shortest AS Path • No import policy on border router • Pick route with shortest AS path • Arbitrary tie break (e.g., smallest router-id) • Performance? • Shortest AS path is not necessarily best • Could have long propagation delay or congestion • Load balancing? • Could lead to uneven split in traffic • E.g., one provider with shorter paths • E.g., too many ties with skewed tie-break

9. Outbound Traffic: Primary and Backup • Single policy for all prefixes • High local-pref for session to primary provider • Low load-pref for session to backup provider • Outcome of BGP decision process • Choose the primary provider whenever possible • Use the backup provider when necessary • But… • What if you want to balance traffic load? • What if you want to select better paths?

10. Outbound Traffic: Load Balancing • Selectively use each provider • Assign local-pref across destination prefixes • Change the local-pref assignments over time • Useful inputs to load balancing • End-to-end path performance data • E.g., active measurements along each path • Outbound traffic statistics per destination prefix • E.g., packet monitors or router-level support • Link capacity to each provider • Billing model of each provider

11. Outbound Traffic: Load, Performance, and Cost • Balance traffic based on link capacity • Measure outbound traffic per prefix • Select provider per prefix for even load splitting • But, might lead to poor performance and high bill • Balance traffic based on performance • Select provider with best performance per prefix • But, might lead to congestion and a high bill • Balance traffic based on financial cost • Select provider per prefix over time to minimize the total financial cost • But, might lead to bad performance

12. Outbound Traffic: What Kind of Probing? • Lots of options • HTTP transfer • UDP traffic • TCP traffic • Traceroute • Ping • Pros and cons for each • Accuracy • Overhead • Dropped by routers • Sets off intrusion detection systems

13. Outbound Traffic: Getting Probes on Paths • Problem • Router selects one path per prefix • How to measure the alternate paths? • Solution #1: special sources (source routing) • Special IP addresses for probe traffic • Router configured to forward probe traffic • Solution #2: special destinations • Special destination servers in various locations • At least one destination per provider AS • Probe traffic sent to each destination

14. Outbound Traffic: How Much Probing? • How often? • Continuously, at some rate • In response to a perceived problem • How diverse of destinations? • Per destination prefix • Just for popular/important prefixes • Select servers throughout the Internet

15. Outbound Traffic: How Often to Change Routes? • ASes with downstream customers • Each change leads to BGP updates • If not, then no new BGP updates occur • TCP flows that switch paths • Out-of-order packets during transition • Change in round-trip-time (RTT) • Impact on the providers • Uncertainty in the offered load • Interaction with their own traffic engineering? • Impact on other end users • Good: move traffic off of congested paths • Bad: potential oscillation as stub ASes adapt?

16. Controlling Inbound Traffic

17. Inbound Traffic: Influencing What Others Do • Harder to control than outbound traffic • IP routing is destination based • Sender determines where the packets go • Control only by influencing others’ decisions • Static configuration of the providers • BGP route attributes sent by the stub • Selective advertising of destination prefixes Provider 1 Provider 2

18. local-pref=100 local-pref=90 traffic Inbound Traffic: Primary and Backup Providers • Ask your provider to be a backup • Provider violates “prefer customer” policy • … by assigning lower local-pref to customer • Backup link is only used if the primary link fails Provider 1 Provider 2 12.34.158.0/24

19. Inbound Traffic: AS Prepending • Make one path look longer • Advertise short path one way • … and longer path another • In the hope of influencing choices • But, how much prepending to do? Provider 1 Provider 2 “12.34.158.024: (3, 3, 3)” “12.34.158.024: (3)”

20. Inbound Traffic: Prepending and Prefer-Cust • Example where prepending doesn’t work • Customer does prepending of AS path • Provider has a “prefer customer” policy • Provider 2 prefers the longer path “12.34.158.024: (1, 3)” Provider 1 Provider 2 “12.34.158.024: (3, 3, 3)” “12.34.158.024: (3)”

21. Inbound Traffic: Programming Your Provider • Better to have selective control over provider • Tell the provider whether to prefer your route • … on a per-prefix basis, with changes over time • Enables adaptive load balancing • … without asking provider to reconfigure policy Provider 1 Provider 2 12.34.158.0/24

22. Inbound Traffic: RFC 1998 on BGP Communities • Provider and customer agree on a “tag” • One tag mean “primary” and the other “backup” • Customer includes tags in BGP advertisements • Provider sets local preference based on tags • BGP community attribute • Opaque attribute with no real meaning • Two numbers: usually AS number and arbitrary number • Sprint example (http://www.sprint.net/policy/bgp.html) • 1239:70 means “assign local pref of 70” • … • 1239:110 means “assign local pref of 110”

23. Example: Tier-1 ISP Setting Local-Preference • Customers • 110: Primary path • 100: Secondary path • 80: Primary backup path • 70: Secondary backup path • Peers • 81-99: In between • Range for traffic engineering Peer Customer

24. Inbound Traffic: Not Enough Prefixes • Stub ASes usually have only a few prefixes • E.g., one prefix, or at most a handful • Not enough granularity to control traffic • Solutions: advertise smaller subnets of prefix • Essentially, create a bunch of smaller prefixes • And apply the load-balancing techniques • Advertise selectively • AS prepending • Communities to set local-pref • …

25. Inbound Traffic: Selective Advertising • Divide the destination prefix • Advertise one subnet to each provider • Advertise the supernet to both providers • Traffic splits due to the longest-prefix match • Supernet ensures backup connectivity after failure Provider 1 Provider 2 12.34.0.0/16 12.34.0.0/17 12.34.0.0/16 12.34.128.0/17

26. Inbound Traffic: Small Subnets, Big Debate • The players • Stub ASes want more control • Advertise smaller subnets • ISPs want to limit table size • Filter BGP advertisements for small blocks • ARIN/RIPE/APNIC • Publish guidelines for acceptable block sizes • Problems • ISPs not getting paid for their routing tables • Risk of network crashes when memory is full • Risk of black-holing a small subnet you filter

27. Project Ideas • Intelligent route-control techniques • Survey approaches to measuring performance • Evaluation of different measurement approaches • Techniques for controlling inbound traffic • Negotiation scheme between ASes • Economic approaches for balancing the tension between fine-grain control and table size • Source routing • Scalable techniques for stub AS to pick the end-to-end route (not just the next-hop AS)

28. Next Class: Convergence Delay • Two papers, intradomain and interdomain • “Analysis of Link Failures in an IP Backbone” • “Delayed Internet Routing Convergence” • Reviews of both papers • Summary • Why accept? • Why reject? • New research directions • Optional NANOG video • “Toward Millisecond IGP Convergence”