Virtual Networking PAVMUG: July 24, 2008

1. Virtual NetworkingPAVMUG: July 24, 2008 Jonathan Butz Services Manager Arraya Solutions, Inc. jbutz@arrayasolutions.com Halim Chtourou Senior Solutions Engineer Arraya Solutions, Inc. hchtourou@

2. Virtual Networking Outline • Arraya Introduction • Virtual Networking • Design Essentials • Design Examples • Advanced Concepts

3. Arraya Solutions, Inc. • IT Infrastructure Consultants since 1999 • Consulting Services in Industry Leading Technologies • Custom Solutions and Services

4. The Arraya Team • Experienced and Knowledgeable • Certified Professionals • Responsive Sales Professionals • Consultative Approach with a Proven Track Record • Flexible • Local Presence and Premier Service • In-house Demo Center, New Data Center • Successful • Consistent Double-Digit Growth Since Inception • Portfolio of Satisfied Reference Customers

5. Satisfied Customers

6. Custom Solutions • Exchange 2007 CCR Design and Migration • Storage architecture, deployment, migration • DR architecture and implementation • VMware architecture and deployment • Health Checks, Report and Recommendations SAN, VMware, Active Directory, Exchange, TSM

7. VMware Solutions • VMware Virtual Infrastructure Partner since 2003 • VMware Authorized Consulting Partner • VMware Premier Partner, VAC Gold Partner • 9 VMware Certified Professionals on Staff • Close Relationships With VMware Team • Planning & Design Accreditation

8. Virtual Networking Outline • Arraya Introduction • Virtual Networking • Design Essentials • Design Examples • Advanced Concepts

9. Physical to Virtual • Increased scale on similar physical footprint • ESX host servicing multiple endpoints • Networking concepts remain the same • Virtual Networking enables additional flexibility

10. Physical to Virtual Physical Switch Physical Switch Virtual Switch

11. Increased Flexibility • Add vSwitches as required • Assign guest OS and physical NICs (vmnics) as required • Guest OS traffic switched internally Virtual Switch Virtual Switch Virtual Switch

12. Virtual Networking Outline • Arraya Introduction • Virtual Networking • Design Essentials • Design Examples • Advanced Concepts

13. Design Essentials • Virtual network topology: same as physical • Conventional access, distribution, core design • Virtual Switches are Access Switches • Isolate certain traffic types where possible

14. Traffic Types • Virtual Machine Traffic • Traffic sourced and received from virtual machines • Traffic between VMs on same vswitch stays internal • VMotion Traffic • Traffic sent when moving a virtual machine from one ESX host to another • Should be isolated from VM traffic • Management Traffic • Should be isolated from VM traffic • Includes heartbeats if VMware HA is enabled • iSCSI Traffic • Should be isolated from all other traffic

15. Virtual Switch Capabilities • L2 Ethernet Switching • VLAN Trunking and Segmentation (802.1Q) • Rate limiting: restrict traffic generated by a VM • VMware NIC Teaming • Load balancing for better use of physical network • Redundancy for enhanced availability • Layer 2 functionality only — no routing • MAC addresses known by registration rather than learned • No MAC learning required • Prevents MAC spoofing

16. VLAN Trunking in ESX • Enables logical network partitioning • Virtual machines connect to virtual switch portgroups • Virtual switch portgroups are associated with a particular VLAN • Virtual switch tags packets exiting virtual machine just as physical switches do for physical servers

17. VLAN Tagging Options VST – Virtual Switch Tagging VGT – Virtual Guest Tagging EST – External Switch Tagging Port Groups assigned to a VLAN vSwitch vSwitch vSwitch vnic vnic vnic vnic vnic vnic vnic vnic vnic VLAN Tags applied in vSwitch VLAN Tags applied in Guest PortGroup set to VLAN “4095” Physical Switch Physical Switch Physical Switch External Physical switch applies VLAN tags Preferred

18. Redundant Paths: Uplinks and Switches NIC Teaming A1 A2

19. Teaming Options for ESX Uplinks • “Originating Virtual Port ID” or “Source MAC” based Teaming • NIC chosen based on originating virtual switch port ID or source MAC • Traffic from the same vNIC sent via same physical NIC (vmnic) until failover • Simple: no link aggregation • “IP Hash” Teaming • NIC chosen based on SRC-DST IP • Link aggregation (EtherChannel) required on physical switch • Limited teaming to single switch except where explicitly supported (Cisco Catalyst 6500 VSS, Nortel Split MLT and some stacked switches) • Better balancing if guest has large number of IP peers • Recommendation: Choose Originating Virtual Port ID based teaming for simplicity and multi-switch redundancy (default)

20. Multiport NICs ESX Host

21. Virtual Networking Outline • Arraya Introduction • Virtual Networking • Design Essentials • Design Examples • Advanced Concepts

22. Design and Network Ports Question • How do I best design the virtual network given VM traffic, VMotion and Management for security and isolation? Answer • Depends on number of physical ports • 4 NIC ports per server recommended, +2 for iSCSI • VLAN trunking highly recommended Design Examples • ESX flexibility allows for multiple variations of valid configurations • Understand your requirements and resultant traffic types and design accordingly

23. Example Infrastructure • 4 ESX Servers • 2 logical groups of virtual machines • VLANs • VLAN 10: Management • VLAN 20: VMotion • VLAN 105: Finance • VLAN 106: Engineering VLANs 10, 20, 105, 106 ESX Host 1 ESX Host 2 VLANs 10, 20, 105, 106 ESX Host 3 VLANs 10, 20, 105, 106 VLANs 10, 20, 105, 106 ESX Host 4 VC Server VLAN 10

24. VLANs 10, 20, 105, 106 ESX with 2 NICs • Create one virtual switch • Connect both physical NICs • Port groups • Port group 10 for Service Console • Port group 20 for VMotion • Port group 105 for Finance VMs • Port group 106 for Engineering VMs • On-board NIC0 (vSwitch1 Uplink) • PG10 (preferred) and PG20 (preferred) • On-board NIC1 (vSwitch1 Uplink) • PG105 (preferred) and PG106 (preferred)

25. ESX with 4 NICs: Option 1 • Create two virtual switches • Connect two physical NICs to each VSwitch • Port groups • Virtual Switch0 • Port group 10 for Service Console • Port group 20 for VMotion • Virtual Switch1 • Port group 105 for Finance VMs • Port group 106 for Engineering VMs

26. Team Team ESX for 4 NICs: Option 1 • On-board NIC0 (vSwitch0 uplink) • PG10 (preferred) and PG20 • On-board NIC1 (vSwitch1 uplink) • PG105 and PG106 • PCI based NIC0 (vSwitch0 uplink) • PG10 and PG20 (preferred) • PCI based NIC1 (vSwitch1 uplink) • PG105 and PG106

27. ESX with 4 NICs: Option 2 • Create one virtual switch • Connect all 4 NICs to VSwitch • Port groups • Port group 10 for Service Console • Port group 20 for VMotion • Port group 105 for Finance VMs • Port group 106 for Engineering VMs • Configure preferred physical NICs • More effective use of available bandwidth • Simplest physical switch configuration: all ports are VLAN Trunks carrying VLANs 10, 20, 105 and 106 SC VMkernel PG105 PG10 PG20 vSwitch vmnic0 1 2 3 Preferred vnic vnic vnic Standby

28. ESX with More than 4 NICs • With Trunks • Use previous approach and scale up to meet additional bandwidth and redundancy requirements • Dedicate NIC pair for iSCSI (if using VM software initiator) • Without Trunks • Dedicate NIC pair for VMotion • Dedicate NIC pair for Service Console • Separate NIC pairs for each network • Dedicate NIC pair for iSCSI (if using VM software initiator)

29. DMZ Architecture • Regulations may require DMZ traffic separation • SOX and HIPPA requirements for isolation are open to interpretation • Many customers dedicate NICs to DMZ traffic • Allows internal and DMZ traffic in same cluster • Compliance may vary by auditor

30. Virtual Networking Outline • Arraya Introduction • Virtual Networking • Design Essentials • Design Examples • Advanced Concepts

31. iSCSI Design • Provides SCSI block storage access over IP network • Relevant for VMs using the iSCSI software-based initiator • Design depends on NIC ports available • General Design Guidance • Keep iSCSI traffic on its own dedicated vlan and subnet • Dedicate NIC pairs to iSCSI traffic • Use teaming as appropriate • “Virtual Source Port ID” setting if all your iSCSI targets share the same IP address • “IP Hash” setting for other scenarios, including the case for multiple targets

32. iSCSI Examples • Two NIC ports • Buy additional NICs if possible • Follow two port example • For high VM traffic • Set SC + VMotion + iSCSI to prefer NIC0 • Set VM traffic to prefer NIC1 • For low VM traffic • Set SC + VMotion to prefer NIC0 • Set VM traffic + iSCSI to prefer NIC1 • Four NIC Ports • Buy additional NICs if possible • Follow two port example • Create additional VSwtich, connect remaining NICs for iSCSI • Six NIC Ports • Follow four port example, dedicate additional NICs to iSCSI

33. Spanning Tree: Not Used by ESX • ESX does not alter STP on physical network • ESX does not participate (does not generate/consume BPDUs) • Use “portfast” or “trunkfast” on physical switch to progress immediately to “forwarding” state • Interconnections between virtual switches are not possible • Loops are not possible within a single virtual switch Virtual Switch Virtual Switch

34. Link-state Tracking: faster failover ESX Host Virtual Switch “Link State Tracking” associates upstream and downstream links

35. C C MACC MACC IPC IPC VMotion: Step by Step ESX Host 1 ESX Host 2 B A MACA MACB IPA IPB VMotion Traffic RARP for MAC move(L2 broadcast to network) MACAMACB MACC MACC Physical Switch Physical Switch

36. Questions? Arraya Solutions, Inc. 521 Plymouth Road Suite 113J Plymouth Meeting, PA 19462 http://www.arrayasolutions.com 866.229.6234 Jonathan Butz Services Manager jbutz@arrayasolutions.com 610.684.8616

37. Hidden Bonus Slides

38. Customer: TCO