430 likes | 924 Views
VXLAN Fundamentals, Architecture & Roadmap. Table of Contents. Data Center IP Fabric ‘Building a strong Foundation’ What is ‘Network Virtualization’? VXLAN Overview VXLAN Packet details VXLAN Terminology VXLAN Host Discovery VXLAN BUM Traffic Handling VXLAN Layer 2 & Layer 3 Terminologies
E N D
Table of Contents • Data Center IP Fabric ‘Building a strong Foundation’ • What is ‘Network Virtualization’? • VXLAN Overview • VXLAN Packet details • VXLAN Terminology • VXLAN Host Discovery • VXLAN BUM Traffic Handling • VXLAN Layer 2 & Layer 3 Terminologies • VXLAN Arista Architecture & Vision • VXLAN Roadmap • VXLAN Visbility
Data Center – ‘IP Fabric’ Building A Strong Foundation
Challenges with current network architecture Oversubscription Legacy Data Center Model • Ports on devices are oversubscribed ~ 8:1 • Higher Oversubscription as traffic traverses north ~ 20:1 North to South Scalability • Scales up and not scales out • Dependent on specific hardware (mix & match) • Not scalable to 40GbE / 100GbE Cost • As multiple layers, it can get $$$ Mobility • What happens if my “IP” changes? • What happens if traffic pattern changes? Layer 2 Domain Layer 2 Domain Layer 2 Domain Layer 2 Domain Latency Multiple points of management, rampant oversubscription, wasteful cost model • High latency • Low predictability Multiple points of management, rampant oversubscription, wasteful cost model
Support for East/West 80:20 traffic pattern Scale up to 64-way ECMP Spine designs All uplinks from ToR are Active/Active Support 100’000s of host ports Non-blocking / Non-oversubscribed architecture Data Center ‘IP Fabric’ • Deploy L3 routing protocols between leaf & spine i.e. BGP, OSPF, or ISIS • Everything is only 3 hops away! • Provide network mobility via ‘Overlay Network’
Arista – Spine/Leaf “IP Fabric” Architecture • Network core is an IP fabric laid out in a Leaf-Spine architecture running ECMP between the two tiers • Leaf switches - Arista 7150-x or 7050Q-x models are deployed at the TOR connecting virtualized servers, bare-metal servers, storage arrays and other devices • Spine switches – Arista 7500’s are deployed at the core • Routing Protocol – Either EGP (BGP) or IGP (OSPF / ISIS) is run in the IP fabric Spine Tier IP Fabric Leaf Tier VTEP1 VTEP3 VTEP4 A 1 B 1 A2 B2 VTEP2 Bare Metal Servers Bare Metal Storage HYPERVISOR 1 HYPERVISOR 2
What is Network Virtualization? Network Virtualization is not the same as Server Virtualization!
Overlays v Underlays Network virtualization: ability to separate, abstract and decouple the physical topology from a ‘logical’ or ‘virtual’ topology by using encapsulated tunneling. This logical network topology is often referred to as an ‘Overlay Network’. Overlay Network Physical Infrastructure i.e. Underlay Network VXLAN disassociates workloads from physical networks, allowing for possible transition to cloud based providers
Types of ‘Overlay’ Technologies Any Overlay technology uses Location & Identity separation Location Identity
Virtual Extensible Local Area Network (VXLAN) • Ethernet in IP overlay network • Entire L2 frame encapsulated in UDP • 50 bytes of overhead • Include 24 bit VXLAN Identifier • 16 M logical networks • VXLAN can cross Layer 3 • Tunnel between ESX hosts • VMs do NOT see VXLAN ID • IP multicast used for L2 broadcast/multicast, unknown unicast • Technology submitted to IETF for standardization • With Arista, Vmware, Red Hat, Citrix, Cisco, and Others Outer MAC DA Outer MAC SA Inner MAC DA InnerMAC SA Optional Inner 802.1Q Original Ethernet Payload Outer 802.1Q Outer IP DA Outer IP SA Outer UDP VXLAN ID (24 bits) CRC VXLAN Encapsulation Original Ethernet Frame
Virtual eXtensible LAN: How does it work? VM-2 10.10.10.2/24 Layer 2 Domain between the VM vWire- VNI 10 VM-1 10.10.10.1/24 VTEP VTEP Subnet-A Subnet-B SW VTEP Encap/Decap VXLAN VTEP HW VTEP Encap/Decap VXLAN Frames MAC&IP are UDP Encapsulated Encapsulation at VTEP node is transparent to IP ECMP fabric
VXLAN Benefits • Feature Benefits • Eliminates current networking challenges in the way of on-demand, virtual environment: • VLAN Sprawl • Single fault domains • Scalability beyond 4096 segments • Proprietary fabric solutions • IP mobility • Physical cluster size and locality • Enables multi-tenancy at scale • Decouples logical networks from physical infrastructure so that applications can be deployed without worrying about physical rack location, IP address or VLAN • Based on open and well known standards
VXLAN Use Cases • Physical to Virtual internetworking • Multi-hypervisor connectivity and integration • Multi-tenant Cloud environments • HA clusters across failure domains • Dynamic growth • Dynamic resource management
VXLAN Packet VXLAN is a MAC-in-IP encapsulation
VXLAN Header VXLAN Header is a 8 Byte field comprising of: • Flags (8 Bits) • VxLAN Network Identifier (VNI) (24 Bits) • Reserved (24 & 8 Bits) – Always set to zero. Reserved (24 & 8 Bits) – Always set to zero. Flags (8 Bits) – I flag is set to 1 for a valid VxLAN Network ID (VNI). The remaining 7 bits (designated "R") are reserved fields and set to zero. VxLAN Network Identifier (VNI) (24 Bits) – Used for identification of the individual VxLAN overlay network on which the communicating VMs are situated. VMs in different VxLAN overlay networks cannot communicate.
VXLAN Terminology – Physical Topology Hardware VTEP Software VTEP Spine Tier IP Fabric Leaf Tier VTI VTI VTEP1 VXLAN Gateway VXLAN Segments VTEP3 VTEP4 VXLAN 10001 A 1 B 1 A2 B2 VTEP2 Bare Metal Servers Bare Metal Storage HYPERVISOR 1 HYPERVISOR 2 VXLAN 10002
VXLAN Terminology – Logical Topology External Host Data Center Network VARP Default Gateway: 10.100.1.1 VARP Default Gateway: 10.100.2.1 VARP Default Gateway: 10.100.2.1 VARP Default Gateway: 10.100.1.1 VXLAN Segment VXLAN Segment VNI VTEP 1 VTEP 3 VTEP 4 VTEP 1 .1 .1 .1 .1 VXLAN 10001 10.100.1.0/24 10.100.2.0/24 VXLAN 10002 .11 .2 .3 .2 .10 .10 B1 B2 A1 A2 Bare Metal Storage Bare Metal Servers
VXLAN Terminology Explained • VTEP: VXLAN Tunnel End Point • VXLAN encapsulation and decapsulation happens at the VTEP • VXLAN Gateway • A device which bridges traffic from VXLAN and non-VXLAN environments. • VXLAN gateways allow for physical and non virtualized devices to communicate with VXLAN networks • A VXLAN gateway can be either a hardware or software device • VNI: Virtual Network Identifier - a 24-bit number is also called the VXLAN segment ID. The system uses the VNI, along with the VLAN ID, to identify the appropriate tunnel. • VXLAN Header – is an 8-byte header that contains the 24-bit VNI value. It lives in between the UDP header and the inner MAC frame being carried over the VTI. • VTI: VTEP Tunnel Interface - a switchport linked to a UDP socket that can be shared between many VLANs. Packets bridged through a vlan into the VTI are sent out the UDP socket with a VXLAN header including a VNI. The socket is bound to a fixed local port, but is not connected to any particular destination port or IP address; logically, we use sendto() (not send()) to transmit VXLAN-encapsulated frames on the socket. Packets arriving on the VTI (via the UDP socket, based on their UDP destination port) are demultiplexed into a VLAN for bridging. A 24-bit VNI within the packet determines which VLAN the packet is mapped to for bridging. • VXLAN Segment - is a Layer 2 overlay network over which VMs communicate. Only VMs within the same VXLAN segment can communicate with each other.
VXLAN Visibility - Arista’s vmTracer • Full physical to virtual visibility • Network audit to ensure reachability • Automated provisioning • Workflow without finger pointing • Other awesome capabilities
Monitoring VXLANs with vmTracer Virtualization • Rapidly correlate vlan to VNI switch5#:show vmtracer vxlan interface Ethernet48 Ethernet48: esx1.aristanetworks.com/ndsTest/dvuplink1 VM Name VLAN vWire Network Multicast -------------------------------------------------------------------------------------------- Exchange 5 Corp 172.20.20.0 239.20.20.0 Apache 6 web 182.10.0.0 220.10.10.0 MySQL 7 ERP 172.20.30.0 239.20.30.0 • view VNIs across the data center from the CLI switch9#:show vmtracer vxlan all 7150s R1: Ethernet 48:esx1/vwTest/dvUplink 1 vWire:Corp -- VLAN:5 vWire:ERP -- VLAN:7 7150s R2: Ethernet 40:esx2/vwTest/dvUplink 1 vWire:Corp -- VLAN:5 vWire:web -- VLAN:6 vmTracer VTEP VTEP VTEP VTEP Hypervisor VMware NSX Physical
OVSDB VNI, VXLAN, VNI ID Automate Learning of VNI State NSX Controller New VNI - CalBears Multicast Group - 224.0.14.13 VNI ID - 650782 Interface Ethernet 24 VXLAN VTEP VNI CalBears Interface Loopback0 VXLAN VTEP Gateway VNI Calbears IP Address 204.181.40.1/24 <--Network VM- Oski VNI - CalBears
VNI ‘Test’: 224.0.0.12 Where is my VM now? spine0: show vmtracer vxlan VNI-Name VNI #VTEPs Learning Mcast Group Status Subnet Auburn 5096 4 Flood 224.0.1.95 Up 204.181.40.0/24 foo 15893425 5 Flood 224.0.4.84 Up 128.218.56.0/24 bar 65456 45 Flood 224.5.1.92 Down 192.168.10.0/20 spine0: show vmtracer vxlan vni Auburn spine0 VNI Name: Auburn VNI Segment ID: 5096 VTEP Type Status Inside Outside Learning Mcast Grp PIM-RP Switch Port Model ESX1 VMware Up 3 VNICs 204.181.21.5 Flood 224.0.1.95 204.181.1.16 ar16 eth15 7050S ar24 Arista Up/GW 204.181.40.1 204.181.1.16 Flood 224.0.1.95 204.181.1.16 ar24 loop0 7150S ar22 Arista Up/Up 1 MAC/IPs 204.181.3.67 Flood 224.0.1.95 204.181.1.16 ar22 eth2 7150S ESX4 VMware Up 4 VNICs 204.181.1.5 Flood 224.0.1.95 204.181.1.16 ar2 eth23 7050T leaf1 leaf2 esx10 esx11 Aubie WarEagle vshield vm-tiger
VNI ‘Test’: 224.0.0.12 Where is my VM now? spine0: show vmtracer interface vxlan Auburn VTEP: ESX1 Role: vSwitch Switch/Port: ar16.foo.com/eth15 Name VNIC Status State IP Address Aubie Network Interface 1 Up/Up vMotion 204.181.40.2 WarEagle Network Interface 2 Up/Up VM-FT-A 204.181.40.3 BooBama Network Interface 1 Up/Down -- 204.181.40.5 VTEP: ar24 Role: Router Switch/Port: ar24.foo.com/loopback0 NAT/PAT Status #ARPs IP Address No Up/Up 45 204.181.40.1 VTEP: ar22 Role: Port-VTEP Switch/Port: ar22.foo.com/eth2 FQDN IP MAC VLAN Status isilon16.foo.com 204.181.40.190 00-00-45-ab-12-fe 5 Up/Up spine0 leaf1 leaf2 128.218.11.x 128.218.10.x esx1 esx11 Aubie WarEagle vshield vm-tiger