Dynamic Connectivity Management with an Intelligent Route Service Control Point

1. Dynamic Connectivity Managementwith an Intelligent Route Service Control Point Kobus van der Merwe AT&T Labs - Research A.Cepleanu, K. D’Souza, B. Freeman, A. Greenberg, D. Knight, R. McMillan, D. Moloney, J. Mulligan, H. Nguyen, M. Nguyen, A. Ramarajan, S. Saad, M. Satterlee, T. Spencer, D. Toll and S. Zelingher

2. Motivation • New wanted and unwanted traffic • VoIP, MMOG • DDoS attacks • New pressures on network operators • Need improved network management practices to dynamically control if/how traffic flows in a network: • Dynamic Connectivity Management INM Sept 2006

3. BGP for connectivity management? • BGP is inherently involved in connectivity • Ideal vehicle to control connectivity • Used to realize a variety of business and/or network management tasks • Typically over slower timescales • But, • BGP is inherently complex • Assembly language like configuration distributed over 100s or 1000s of routers • Changing dynamically, challenging (if possible) • Lack of direct control • Influence decision process indirectly • Set attributes locally to influence decision process on a remote router • Network unaware • Decision making not influenced by dynamic changes in network INM Sept 2006

4. IRSCP for Dynamic Connectivity Management • Intelligent Route Service Control Point (Formerly: Route Control Platform) • Logically centralized BGP speaking network entity: • Control plane only function, separate from routers • Connectivity control at protocol timescales • Raise level of abstraction to simplify connectivity management tasks • Allows connectivity management to be influenced by external network intelligence • Sampling of connectivity management applications • Illustrate approach + research direction INM Sept 2006

5. Operator Input Network Intelligence IBGP Control Monitoring IRSCP R R RR RR R R IRSCP • Logically centralized, speaks iBGP (phase-1 RCP) • Routers still make own decisions + information hiding • Initially deploy in parallel with route-reflector hierarchy • Direct operator and/or (controlled) customer input • Network intelligence input • Routing decisions influenced by external information INM Sept 2006

6. Wanted PE PE PE Attack IBGP PE5 Physical link PE4 PE3 DDoS Detection PE2 IRSCP CE Selective DDoS blackholing • Pre-configure blackhole static route to /dev/null on all PEs • When detect DDoS attack, advertise very specific route to attack target with next-hop pointing to BH static route • Only to PEs where there is attack traffic • DDoS attacks are not that distributed (LSAD DDoS analysis paper) • Network intelligence: DDoS detection system • Still collateral damage, good traffic on that PE also dropped • BGP speaking IRSCP too coarse grained • Rudimentary form of type of control possible with non-BGP speaking IRSCP (LSAD PRIMED paper) INM Sept 2006

7. Other ISP PEiii PEi PEii IBGP PE5 Physical link PE4 PE3 PE6 PE2 IRSCP CE PE1 PE7 Planned Maintenance Dryout • Goal: move traffic off network element that will undergo planned maintenance (PM) without any impact (I.e., no session resets) • With IRSCP, if alternate path exists • Multihomed customers/data centers (multihomed to IRSCP capable provider) • Multiple peering sessions • Two directions, traffic going to PM element + traffic coming from PM element • Towards PM element • Steer traffic away by making alternate path(s) more preferable (e.g., by increasing the local preference attribute) • From PM element: • Convince network elements on “other side” of PM element to steer traffic away • Data center case: pre-configure policy on CE, routes with special community value less preferred • Peering case: advertise routes with lower MED via desired peering points • Network intelligence: planned maintenance event, alternate paths, load INM Sept 2006

8. Site E Site D CE4 CE5 GW GW IBGP PE5 PE4 Physical link P1 P2 IRSCP P3 PE1 PE3 PE2 CE1 Site A Site C CE3 CE2 Site B VPN Gateway selection • Problem: MPLS VPNs, two routes to the same destination, PE breaks tie based on IGP distance • E.g., two default routes for Internet gateways • Tie breaking reasonable in terms of delay, not so in terms of VPN customer needs • E.g., east & west gateways, utilize both under normal conditions • IRSCP: allow customer to dictate policy of which sites should be using which GW + backup strategies • IRSCP adjust preference of routes based on these policies • Adjusting local preference settings • Allow customer to dynamically change policies (Web portal) • Can be done dynamically based on GW load + load on ingress • Network Intelligence: customer policy, load information INM Sept 2006

9. AS 1 PEiii PEi PEii IBGP PE5 PE4 Physical link PE3 PE6 PE2 P1 P2 CE Customer Network IRSCP P3 PE1 PE7 AS 0 Network aware load balancing • Problem: two routes to the same destination, PE breaks tie based on IGP distance • E.g., customer/data center dual homed to provider • Depending on offered load, egress links completely unbalanced • Common problem: 72% multihomed customers,one links carries all traffic (red curve) • IRSCP: selects routes for ingress so as to balance traffic • Take offered load, egress load and IGP distance into account during route selection • Simulation: offered load balanced on per prefix/per ingres PE basis (green curve) • 25% still unbalanced, 50% customers link ratio 0.85 or better, for top 30% ratio is 0.98 or better • Network Intelligence: balancing policy, offered + egress load information, IGP INM Sept 2006

10. Implementation Operator External Information Application Logic • IRSCP Core • Sends/receives BGP messages • Current implementation built on Quagga • Application logic • Deals with “network intelligence” • Invokes high level interface on IRSCP • Glue logic • Convert high level commands to detailed configuration IRSCP Glue Logic IRSCP Core Network Elements INM Sept 2006

11. Raising level of abstraction Operator/application logic interface (what rather than how): • Selective blackholing --cmd addblackhole --routerlist PE1,..PEn, --prefix • PM dryout --cmd adddryout --dryout PE1 --backup PE2 • VPN gateway selection/Load balancing --cmd addgroup --ingress PEa --group N (--vpn VPNA) --cmd addpolicy --egress PEb -- group N INM Sept 2006

12. Site E Site D CE4 CE5 GW GW IBGP PE5 PE4 Physical link P1 P2 IRSCP P3 PE1 PE3 PE2 CE1 Site A Site C CE3 CE2 Site B VPN Gateway selection - Example • Ensure PE1 selects default route via PE4 with highest priority and use route via PE5 as backup • addgroup --ingress PE1 --vpn VPNA --group 1 • addpolicy --egress PE4 --vpn VPNA --group 1 --prefix DEFAULT --pref 110 • addpolicy --egress PE5 --vpn VPNA --group 1 --prefix DEFAULT --pref 100 INM Sept 2006

13. addpolicy --egress PE4 --vpn VPNA --group 1 --prefix DEFAULT --pref 110 Example glue logicVPN gateway selection addgroup --ingress PE1 --vpn VPNA --group 1 • route-map vpnin-a permit 1 • match ip next-hop PE4 • match extcommunity VPNA • match ip address prefix-list DEFAULT • set community GRP1_LP110 additive • on-match next • ! • route-map vpnout-b permit 1 • match extcommunity VPNA • on-match goto 3 • ! • route-map vpnout-b permit 3 • match ip peeraddress PE1 • on-match goto 6 • ! • route-map vpnout-b permit 6 • match community GRP1_LP100 • set local-preference 100 • ! • route-map vpnout-b permit 7 • match community GRP1_LP110 • set local-preference 110 • ! VPN Selection Per VPN PE Selection • Similarly for other addpolicy command • Works well • Kind of ugly! • Room for improvement Group 1 Policy Selection INM Sept 2006

14. Conclusion • Dynamic Connectivity Management Applications • Various stages of trial deployment • Work in progress • High level abstraction desirable • Exploring alternative implementations of IRSCP core and glue logic • IOS like CLI, right abstraction abstraction? • More applications in the pipeline INM Sept 2006