1 / 20

Alleviating Constraints with Resource Pools & Live Migration with Enhanced VMotion*

Alleviating Constraints with Resource Pools & Live Migration with Enhanced VMotion*. Breakout Session# 2823 Raghu Yeluri Sr. Architect Intel Corporation Date: 09/16/2008 . Alan Bumgarner Program Manager Intel Corporation . Jim Blakley Director, DC Virtualization

nathan
Download Presentation

Alleviating Constraints with Resource Pools & Live Migration with Enhanced VMotion*

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Alleviating Constraints with Resource Pools & Live Migration with Enhanced VMotion* Breakout Session# 2823 Raghu Yeluri Sr. Architect Intel Corporation Date: 09/16/2008 Alan Bumgarner Program Manager Intel Corporation Jim Blakley Director, DC Virtualization Intel Corporation

  2. Contents • Problem • VMotion* Overview • VMotion* CPU Compatibility Constraints • Why we care Solution: “Enhanced VMotion*” & Intel® VT FlexMigration Case Study: Enabling “Enhanced VMotion*” with Multi-Generation Intel Processors • Wrap Up

  3. Core Enabling Technology for Virtualization 2.0 Use cases: Dynamic Load balancing, planned maintenance/server upgrades, High Availability, Power-aware load balancing Key Requirements for Migration Part of the same Datacenter (should have the same subnet) Connected to the same storage (SAN, NAS, iSCSI) Recommended: dedicated Gigabit network for VMotion Destination host must have enough resources Source and destination hosts must have compatible CPU models. VMotion* Overview Capability provided by VMware to migrate a live Virtual Machine (VM) from one physical server to another. Key Requirement: Source and destination hosts must have compatible CPU models

  4. VMotion* Steps VM running on server esx01 to be moved to esx02 Resume VM on esx02 1 4 Provision VM on esx02, pre-copy memory & CPU state from esx01 to esx02, log changes to a bitmap 2 5 Delete virtual machine from esx01 VMotion Successfully Completed… 3 Suspend the VM on esx01 and copy changed memory to esx02 Memory and CPU State copied to the target host!

  5. VMotion* requires all processors in “Live VM Migration Pool” be “compatible” Compatible = same ring 3 visible instructions (e.g. visible to all application software) E.g. 64-bit, SSE3, SSE4.1, SSE4.2, etc Caches, speeds, #of cores, #of sockets can vary CPU Vendor has to be same (not the OEM) Why? User mode features are not intercepted… execute directly on the CPU from the application. If an application starts using an instruction… …it wants to keep using it even when an application is migrated to a new server If the instruction suddenly isn’t present…the application “breaks Example 1: SSE4.1 (Penryn) instruction support Running application which uses SSE4.1 Migrate to a CPU which does not support SSE4.1 (Merom) => crash Why Compatible CPUs?

  6. X X Why we care: DC Environment** for VMotion* Compatibility Pool of Compatible Machines VMotion VMotion 4S+ 2S 1S 4S 2S 1S 4S+ 2S 1S Dual-Core, Quad-Core Multi-Core Quad-Core Dual-Core Multi-Core 45nm+ Future Intel (SSE4.2. Nehalem & Beyond) 45nm (SSE4.1) Intel Core2™ Micro architecture (Penryn) 65nm (SSE3) Intel Core2™ Micro architecture (Merom) Multiple Resource Pools. Data Center Fragmentation ** Shows a DC environment with Intel-based Processors.

  7. Why we care: Impact of Multiple Resource Pools • Islands of Clusters/Pools, Limited Fungibility of resources • In-effective utilization of resources; Cannot leverage available capacity in a different Pool • Reduced benefit of VMotion* & DRS • Cannot add newer servers incrementally to a VMotion compatible pool • Cannot take advantage of the faster performance and lower power processors.. • Customers need simpler ways to determine if new hardware are compatible with current environment

  8. What: Capability in VMWare ESX3.5U2* to increase VMotion* Compatibility Automaticallyensures hosts in a cluster present same CPU feature set, even with different generation CPUs on hosts. (CPUs must be from the same vendor) Will enable resource pools to include hosts with multiple generation of processors How: Defines extension to a resource cluster called Enhanced VMotion Compatibility (EVC). Defines baseline feature/instruction set for cluster that host processors present to VMs. For Intel Processors: Baseline feature set: Intel Core Micro Architecture (Merom-based) - SSE3 Leverages Intel® VT FlexMigration Assist to present baseline feature set to VMs. Hosts with out ‘baseline’ feature set or VT FlexMigration hardware assist – cannot be added to the EVC cluster. EVC capability can be enabled/disabled from Virtual Center Solution: Enhanced VMotion*

  9. Intel® VT FlexMigration Assist 2006-2007 2007-2008 2008-2010+ Intel® Core™ (SSE3) 45nm Intel®Core™ (SSE4.1) Next Generation Intel (SSE4.2) µarch Penryn based Nehalem based Merom based Xeon 7300 Xeon 7400 Future 4P+ Dual Core 7200 Xeon 5300 Xeon 5400 Future 2P Dual Core 5100 Dual Core 5200 Xeon 3200 Xeon 3300 Future 1P Dual Core 3000 Dual Core 3100 Solution: Intel® VT FlexMigration Built-in hardware assist starting in 45nm Intel® Core™ Micro architecture that increases live migration compatibility pool • VT-based VMMs (64bit VMWare* ) : “Intel VT-x execution control” • Non VT-based VMMs (BT – 32Bit VMWare*) : “CPU-ID Masking” Allows VMWare’s ESX to report a consistent set of available instructions to guest software running within a VM. Investment Protection and Future Proofing Resource Pools

  10. Case Study: Enhanced VMotion* with multi-generation Intel Processors vConsolidate Consolidate Stack Unit (CSU) • Proof of Concept Goal: • Test ride and demonstrate Enhanced VMotion* & Intel® VT Flex Migration • Identify integration & configuration issues to use Enhanced VMotion* Approach: • Realistic VMs & workloads – vConsolidate benchmark • VConsolidate 1.1 • 2 CSUs (10VMs) • CSU – 5 VMs (DB, Java, Web, Mail, Idle) • EVC Cluster with multi-generational Intel® Xeon® processor hosts • Intel® Core™ (merom) and Intel® Core2™ (penryn) micro architecture servers • Manual & Automatic Migration with VMware DRS*

  11. Initial Hosts • - Added later 1 2 Case Study : EVC Cluster setup Baseline Feature set: SSE3 (Merom) 2 2 Intel® Xeon® 3200 (1S/4C, Merom-based) (Second OEM) Intel® Xeon® 7400 (4S/6C, Penryn-based) 1 1 1 Intel® Xeon® 5365 (2S/4C, Merom-based) Intel® Xeon® 5460(2S/4C, Penryn-based) Intel® Xeon® 7350(4S/4C, Merom-based) EVC Cluster with Intel® Core™ (Merom) and Intel® Core2™ (Penryn) architecture Servers

  12. EVC Cluster setup: Adding physical hosts • Issue 1: In compatible hardware. • Certain CPU features not enabled on physical host • Execute Disable (XD) Bit • VT flag • Fix: Enable XD Bit and VT flag from the BIOS. • Need an AC recycle for the VT flag to take into effect.

  13. Issue 2: Having active VMs on a host in the cluster • Fix 1: Power down all the active VMs on the physical host. • Fix 2: Migrate VMs to a host with same baseline feature set, Power-down VMs, add host to the cluster and migrate VMs back to the cluster. EVC Cluster setup: Adding physical hosts -2 • Issue 3: Incompatible host being added into a EVC Cluster. • All the hosts in a EVC cluster should have the ESX3.5U2 software. • Fix: Upgrade the host to ESX3.5 U2

  14. DB DB Java Java Idle Mail Web Web Idle Mail DB DB Java Java Idle Mail Web Web Idle Mail Java Java Java Java Idle Idle DB DB DB DB Mail Mail Web Web Idle Idle Mail Mail Web Web DB Java Java Idle DB Web Idle Mail Mail Web Case Study: Migration Results After Enhanced VMotion* Before Enhanced VMotion* Intel Xeon 5460 (2S/4C, Penryn-based) Intel Xeon 5365(2S/4C, Merom-based) Intel Xeon 5460(2S/4C, Penryn-based) Intel Xeon 7350(4S/4C, Merom-based) Manual Migration DRS: Conservative DRS: Medium DRS: Aggressive Successful migration between multi-generation Intel-based servers within the same pool!!

  15. EVC Requirements - Summary • All hosts in the cluster should have ESX3.5 U2. • Intel VT and Execute Disable (XD) flag needs to be enabled in BIOS for systems in EVC enabled clusters. • Hosts should have Intel ®Xeon® 3200, Intel ®Xeon® 5300, Intel ®Xeon® 7300 or later processors. • No VMs should be active on a host when being added to the EVC cluster. • EVC can be enabled/disabled from Virtual Center. • Need the latest version of Virtual Center (VC2.5 U2) to enable EVC.

  16. Wrap Up • VMotion* enabling new usages – HA, Power-aware load-balancing, etc. • CPU compatibility requirements of VMotion* creating Islands of clusters/pools – DC fragmentation • Enhanced VMotion* addresses “Multiple Resource Pool Constraints” in Data Center. • For Intel Processors: Enhanced VMotion* leverages Intel® VT FlexMigration hardware assist. • Intel® VT FlexMigration provides Investment Protection and future proofing of resource pools

  17. Backup

  18. Creating Enhanced VMotion Compatible clusters

  19. Create a regular resource cluster from VI client; You will not see the ‘Enhanced VMotion’ options during the creation of the resource cluster. “Edit Settings” to see ‘VMWare EVC’ option and select ‘Enable EVC for Intel Hosts’ Decide baseline ‘feature set/instruction set’ for the cluster – Default: “Merom-based” processors Advanced Usage: Change the baseline instruction set Create a new cluster and enable for Enhanced VMotion

  20. Issue 1: In compatible hardware in the cluster for Enhanced VMotion. • Certain CPU features not enabled • Execute Disable (XD) & VT flag Disabled with 32bit OS and ESX3.5 • Fix: From the BIOS, enable the XD and VT flags.. • Issue 2: Having active VMs on a host in the cluster • Fix: Power down all the active VMs on the physical host. • Migrate VMs to a host with same baseline feature set, VMotion VMs, migrate back to cluster once setup • Issue 3: Incompatible Host being added into a EVC Cluster. • All the hosts in a EVC cluster should have the ESX3.5U2 software. Upgrade host to ESX3.5 U2 Enabling EVC for an existing resource cluster

More Related