SDN Controller Challenges

1. SDN Controller Challenges

2. The Story Thus Far • SDN --- centralize the network’s control plane • The controller is effectively the brain of the network • Controller determines what to do and tell switches how to do it.

3. The Story Thus Far

4. The Story Thus Far Something Happened!!!!

5. The Story Thus Far Let’s Ask the Brian!!!!

6. The Story Thus Far Think about what happen… Maybe come up with a solution

7. The Story Thus Far • Controller runs control function • Control function creates switch state • F(global network state)  Switch state • Global network state can be graph of the network Tell the network what to do

8. Challenges with Centralization • Single point of failure • Fault tolerance • Performance bottleneck • Scalability • Efficiency (switch-controller latency) • Single point for security violations

9. Motivation for Distributed Controllers • Wide-Area-Network • Wide distribution of switches: from USA to Australia. • High latency between one controller and All switches • Application + Network growth • Higher CPU load for controller • More memory for storing FIB entries and calculations • High availabilit

10. Class Outline • Fault Tolerance • Google’s B4 paper • Controller Scalability • Ways to scale the controller • Distributed controllers: Mesh Versus Hierarchy • Implications of controller placement

11. Fault Tolerance

12. Google’s B4 Network • Provides connectivity between DC sites • Uses SDN to control edge switches • Goal: high utilization of links • Insight: fine-grained control over edge and network can lead to higher utilization • Distributed Controllers • One set of controllers for each Data center (site)

13. Google’s B4 Network • Provides connectivity between DC sites • Uses SDN to control edge switches • Goal: high utilization of links • Distributed Controllers • One set of controllers for each Data center (site)

14. Fault Tolerance in B4 • Each site runs a set of controller • Paxos is run between controllers in a site to determine master

15. Quick Overview of Paxos • Given N controllers • 1 Acts as leader, and N-1 as workers • All N controller maintain the same state • Switches interact with leader • Change doesn’t happen until whole group agrees • Failure of primary • N-1 work together to elect a new leader(determine new leader) Propagate State changes Network Events

16. Pros-Cons of Paxos • Pros • Well understood and studied; gives good FT • Many implementations in the wild • E.g. Zookeeper • Cons • Time to recover • Impacts through of the put of the entire system

17. Controller Scalability

18. What limits a controller’s scalability? • Number of control messages from switch • Depends on the application logic • E.g. MicroTE/Hedera periodically query all switches for stats • Reactive controller, evaluated in NoX, requires each switch to send messages for a new flow • Packet-in (if reactive Apps) • Flow stats, Flow_time-outs

19. What limits a controller’s scalability? • Application processing overhead • The controller runs a bunch of application • Similar to: A server running a set of programs • CPU/Memory constraint limit how the app runs

20. What limits a controller’s scalability? • Distance between controller and the switches Hedera L3 FW Controller 1

21. How to Scale the Controller. • Obvious: add more controllers. • BUT: how about the applications? • Synchronization/concurrency problems. • Who controls which switch? • Who reacts to which events? Hedera L3 FW Hedera L3 FW Hedera L3 FW ? ? Controller 1 Controller 2 Controller N Stats + Install OF entries

22. Medium Sized Networks • Assumption: • controller can’t store all forwarding table entries in memory • But can process all events and run all apps • Each controller • Get same network events+ running same app.  same output • But store output for only a fraction and config only a fraction Hedera L3 FW Hedera L3 FW Hedera L3 FW Controller 1 Controller 2 Controller N Stats + Install OF entries

23. Medium Sized Networks: hyperflow • Each controller • Push state to each controller • Each controller things it’s the only one in the network Sub-subscribe ssytem Hedera L3 FW Hedera L3 FW Hedera L3 FW Controller 1 Controller 2 Controller N Stats + Install OF entries

24. Large Sized Networks • Assumptions • Each controller can’t store all the FIB entries • Each controller can’t run the entire application or handle events • Need to partition the application • But how?

25. Application partition 1 • Approach 1: each controller runs a specific application • How do your resolve conflicts in FW entries • Apps can conflict in the rules they install Hedera L3 FW Controller 1 Controller 2 Controller N

26. Application partition 2 • Approach 2: all controllers run the same application but for a subset of devices • Results in a Distributed Mesh control plane Abstract Network view Hedera L3 FW Controller 2 Hedera L3 FW Hedera L3 FW Controller 1 Controller N

27. Application Partition 2 • Abstract view exchanged with each other • Abstract view reduces the n/w information used by each controller REAL NETWORK Hedera L3 FW Controller 2 Abstraction Provided by Controller N Abstraction Provided by Controller 1 Controller 2’s View of NETWORK

28. ONIX to the SDN Programmer • Controllers synchronize through a DB or DHT • So each app needs synchronization code. • How do you deal with concurrency. • How to synchronize between domains. • How many domains? Or controllers? • How many switches in a domain?

29. Application partition 3 • Approach 3: divide application into local, and global. • Results in a hierarchical control plane • Global Controller and Local Controllers • Applications that do not need network-wide state • Can be run locally without communicate with other controllers

30. Are Hierarchical Controllers Feasible • Examples of local applications: • Link Discovery, Learning switch, local policies • Examples of local portions of a global algo • Data center Traffic engineering • Elephant flow detection (hedera) • Predictability detection (MicroTE) • Local apps/controllers have other benefits • High parallelism • Can be run closer to the devices.

31. Kandoo: Hierarchical controllers • 2 levels of controllers: global and local • Local applications are embarrassingly parallel • Local shields global from network events Hedera Global Controller Hedera L3 FW Hedera L3 FW Hedera L3 FW Controller 2 Controller 1 Controller N

32. Kandoo: Hierarchical controllers • Local Controllers: run local apps • Returns abstract view to the global controller • Reduces # events sent to global and reduce size of network seen by Hedera Global Controller Hedera L3 FW Hedera L3 FW Hedera L3 FW Controller 2 Controller 1 Controller N

33. Kandoo: Hierarchical controllers • Global Controllers • Runs global apps: AKA apps that need network wide state Hedera Global Controller Hedera L3 FW Hedera L3 FW Hedera L3 FW Controller 2 Controller 1 Controller N

34. Hedera Reminder • Goal: reduce network contention • Insight: contention happens when elephants share paths. • Solution: • Detect Elephant flows • Place Elephant flows on different flows

35. Implementing Hedera in Onix • 2 levels of controllers: global and local • Local applications are embarrassingly parallel • Local shields global from network events Exchange TM+detection Hedera: detection +placement Hedera:detection+placement Controller 1 Controller 2 Flow Table entries Flow Table entries Stats Stats

36. Implementing Hedera in Kandoo • Local Controllers: get stats from networks + elephant detection • Global Controller: decide flow placement + flow installation Hedera: Global placement Global Controller Install new flow table entries Inform of elephant flows Elephant detection Elephant detection Elephant detection Controller 2 Controller 1 Controller N Stats

37. Implementing B4 in Kandoo like architecture • Local Controllers: get stats from networks + determines demand • Global Controller: calculate paths for traffic Install TE Ops TE+BW allocator TE DB Global Controller Inform of Flow demands Elephant detection Elephant detection Elephant detection Site Controller 2 Site Controller Site Controller N Stats + Install OF entries

38. Kandoo to the SDN Programmer • Think of what is local and what is global • When apps are written, annotate with local flag • Kandoo will automatically place local • And place global. • Kandoo restricts messages between global and local controllers • You can’t send OF styles messages • Must send Kandoo style messages

39. Summary • Centralization provide simplicity at the cost of reliability and scalability • Replication can improve reliability and scalability • For Reliability, Paxos is an option • For Scalability, conqueror and divide • Partition the applications • Kandoo: Local apps and global apps • Partition the network • Onix: each controller controls a subset of switches (Domain)