Diverter: A New Approach to Networking Within Virtualized Infrastructures

1. Diverter: A New Approach to Networking Within VirtualizedInfrastructures Aled Edwards, Anna Fischer, Antonio LainHP Labs

2. Outline • Data Center Networks for Cloud Computing • Our Approach: Diverter • Evaluation • Future Work

3. Data Center Networks for Cloud Computing

4. Data Center Networks for Cloud Computing Goals (and Challenges!) • Multi-tenancy and Security • Host multiple customers on a single shared infrastructure • Allow each customer to configure their own network topology to suit application needs • Data and performance isolation between customers, and the utility • Allow controlled and efficient inter-communication between customers if required and permitted • “provide rich ecosystem of interacting services” • Large scale • Automation • Flexibility / Programmability • Performance

5. Data Center Networks for Cloud Computing Problems with Traditional Approaches • Traditional L2 • Flat network: isolation, scalability • VLANs: configuration, management • Encapsulation, Tunneling • Explicit routing entities required, e.g. routing VMs • Traditional L3 • Mobility • Routing bottlenecks

6. Our Approach:Diverter

7. Our Approach: Diverter High-level Overview • Isolate customer resources into Cells • Cell is a collection of virtual resources • Cell has a single owner • Each Cell can have its own virtual network topology • Cells consist of several Subnets • Cell owner can define network policies • Security: define who can communicate with VMs • QoS: define bandwidth limits for VMs

8. Our Approach: Diverter Virtual Network Topology Globally managed virtual IP address space representing virtual network topologies IP address format: 10.<CELL>.<SUBNET>.<HOST> (for example) Subnet C3 Subnet A1 Subnet C2 Subnet B2 Subnet A2 Subnet B1 Subnet C1 Cell A Cell C Cell B Virtual Router Virtual Router Virtual Router

9. Our Approach: Diverter Realisation as a Distributed Virtual Router • Virtual routers are realised as Distributed Virtual Router implementation (“VNET”) • VNET component running on each server • VNET intercepts packets to/from VMs, processes them, eventually forwards them, or discards them • VNET takes care of • Simulating routing across subnets, or Cells • Multicast/broadcast distribution • Address discovery As virtual routing functionality is distributed across all servers rather than implemented by particular, traditional routing entities, communication between any endpoints in the infrastructure always involves just a single network “hop”.

10. Our Approach: Diverter How Does It Work? MAC Rewriting! • VNET rewrites packets to simulate routing hop • Packets are sent to / received from virtual router interface when crossing subnets • Important to emulate behaviour of traditional network topology • VNET uses (modified) ARP to discover physical machines hosting a particular VM • VNET rewrites packets to send directly to physical machines hosting destination VM • VNET rewrites packets to limit VM broadcast/multicast traffic to particular Cell/subnet

11. MAC Rewriting Simplified 1. Packet TX Virtualmachines 7. Packet RX Virtual machines • Direct network hop between any endpoint • No virtual MACs leaking onto the physical wire 2. Packet intercept 6. Packet RW Physical host B Physical host A 3. Packet RW 4. Packet TX 5. Packet RX sVMAC dVMAC sVMAC dVMAC sPMAC dPMAC sPMAC dPMAC Physical network

12. Virtual Router Simulation 3. Packet TX Virtualmachines 9. Packet RX Virtual machines DHCP Response with Virtual Router IP 2. ARP Request / Reply for Router IP 4. Packet intercept Virtual MACs do not leak across subnets! 8. Packet RW Physical host B Physical host A 5. Packet RW 6. Packet TX 7. Packet RX sVMAC RVMAC sPMACdPMAC sPMAC dPMAC RVMAC dVMAC Physical network

13. Our Approach: Diverter Further Benefits • Efficiency • Use of multicast/unicast ARP instead of broadcast • Local DHCP response generation • No packet encapsulation • Fast tracking of moving VMs/addresses • Security • Integrated network policy framework • Enforcement of fine-grained packet filtering • Allow frequent changes of network policies • Manageability • No programming of physical infrastructure required • No synchronization between physical switches and servers • Only rely on underlying flat L2 network • Separation of concerns: network administrators vs. server administrators • Communication possible with non-VNET servers • No programming of explicit routing entities required • No specific hardware (or hardware modifications) required

14. Evaluation

15. Traditional L2 vs. Diverter Intra-subnet vs. Inter-subnet Communication Subnet B SubnetA Routing VM Subnet A Traditional L2 Diverter Physical network

16. Performance Evaluation VM Network Throughput

17. Future Work

18. Future Work • Direct Network I/O • Integrate with virtualization-aware HW on server-side, e.g. SR-IOV NICs, blade server networking • Integration with new I/O virtualization approaches developed around KVM/Xen • QoS • Virtual Network Cloning • Data Center Network Federation • L2 Scalable Data Center Ethernet