1 / 28

Steve Hand

Xen and the Art of Virtualization. Steve Hand Keir Fraser, Ian Pratt, Christian Limpach, Dan Magenheimer (HP), Mike Wray (HP), R Neugebauer (Intel), M Williamson (Intel).

collin
Download Presentation

Steve Hand

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Xenand the Art of Virtualization Steve Hand Keir Fraser, Ian Pratt, Christian Limpach, Dan Magenheimer (HP), Mike Wray (HP), R Neugebauer (Intel), M Williamson (Intel) Computer Laboratory

  2. Talk Outline • Background: XenoServers • The Xen Virtual Machine Monitor • Architectural Evolution • System Performance • Subsystem Details… • Status & Future Work

  3. Global services and apps Exploit network topology Open, commercial infra. Incremental rollout Flexible platform Unified management XenoServers Deploy XenoCorp XenoServer Client

  4. Architecture components • Brokers (e.g. XenoCorp) • Intermediary between user and merchant • Resource search services (e.g. XenoSearch) • Suggest ‘best’ XenoServer(s) for a job • Other high-level services and libraries • Distributed file systems, control interface, resource management wrappers • Execution platform (XenoServer) • Securely partition and account resource usage

  5. Architecture components • And don’t mandate single OS image… • And use commodity (viz. x86) hardware… • But retain high performance! These goals led to the design and implementation of the Xen hypervisor • Execution platform (XenoServer) • Securely partition and account resource usage

  6. XenoServers vs. GRID

  7. Requirements • Isolation for competing services: • Secure protection • Strong resource isolation and QoS • CPU, memory, disk space and b/w, network b/w • Accounting and logging • Flexible execution environment • Linux, NetBSD, Windows XP, … • Support many services per machine • Only limited by the service’s resource needs • Service-based management incl. migration

  8. Virtualization Options • Single OS image: Ensim, Vservers, CKRM • Group user processes into resource containers • Hard to get strong isolation • Full virtualization: VMware, VirtualPC • Run multiple unmodified guest OSes • Hard to efficiently virtualize x86 • Para-virtualization: UML, Xen • Run multiple guest OSes ported to special arch • Arch xen/x86 is very close to normal x86

  9. Xen 2.0 Features • Secure isolation between VMs • Resource control and QoS • Only guest kernel needs to be ported • All user-level apps and libraries run unmodified • Linux 2.4/2.6, NetBSD, FreeBSD, WinXP • Execution performance is close to native • Live Migration of VMs between Xen nodes • Xen hardware support: • SMP; x86 / x86_64 / ia64; all Linux drivers

  10. Xen 1.2 Architecture Domain 0 Domain 1 Domain 2 Domain 3 Unmodified User-Level Application Software Ported ‘Guest’ Operating Systems Xen Hypervisor Hardware

  11. Xen 2.0 Architecture Domain 0 Domain 1 Domain 2 Domain 3 Unmodified User-Level Application Software Ported ‘Guest’ Operating Systems Xen Hypervisor Hardware

  12. System Performance 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 L X V U L X V U L X V U L X V U SPEC INT2000 (score) Linux build time (s) OSDB-OLTP (tup/s) SPEC WEB99 (score) Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

  13. Xen Para-Virtualization • Arch xen/x86 – like x86, but replace privileged instructions with Xen hypercalls • Avoids binary rewriting and fault trapping • For Linux 2.6, only arch-dep files modified • Modify OS to understand virtualised env. • Wall-clock time vs. virtual processor time • Xen provides both types of alarm timer • Expose real resource availability • Enables OS to optimise behaviour

  14. x86 CPU virtualization • Xen runs in ring 0 (most privileged) • Ring 1/2 for guest OS, 3 for user-space • GPF if guest attempts to use privileged instr • Xen lives in top 64MB of linear addr space • Segmentation used to protect Xen as switching page tables too slow on standard x86 • Hypercalls jump to Xen in ring 0 • Guest OS may install ‘fast trap’ handler • Direct ring user-space to guest OS system calls • MMU virtualisation: shadow vs. direct-mode

  15. MMU Virtualizion : Shadow-Mode guest reads Virtual → Pseudo-physical Guest OS guest writes Accessed & Updates dirty bits Virtual → Machine VMM Hardware MMU

  16. MMU Virtualization : Direct-Mode guest reads Virtual → Machine guest writes Guest OS Xen VMM Hardware MMU

  17. Para-Virtualizing the MMU • Guest OSes allocate and manage own PTs • Hypercall to change PT base • Xen must validate PT updates before use • Updates may be queued and batch processed • Validation rules applied to each PTE: 1. Guest may only map pages it owns* 2. Pagetable pages may only be mapped RO • Xen tracks page ownership and current use • L4/L3/L2/L1/Normal (plus ref count)

  18. MMU Micro-Benchmarks 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 L X V U L X V U Page fault (µs) Process fork (µs) lmbench results on Linux (L), Xen (X), VMWare Workstation (V), and UML (U)

  19. I/O Architecture • Xen IO-Spaces delegate guest OSes protected access to specified h/w devices • Virtual PCI configuration space • Virtual interrupts • Devices are virtualised and exported to other VMs via Device Channels • Safe asynchronous shared memory transport • ‘Backend’ drivers export to ‘frontend’ drivers • Net: use normal bridging, routing, iptables • Block: export any blk dev e.g. sda4,loop0,vg3

  20. Device Channel Interface

  21. 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 L X V U L X V U L X V U L X V U Tx, MTU 1500 (Mbps) Rx, MTU 1500 (Mbps) Tx, MTU 500 (Mbps) Rx, MTU 500 (Mbps) TCP bandwidth on Linux (L), Xen (X), VMWare Workstation (V), and UML (U) TCP results

  22. Isolated Driver VMs

  23. Live migration for clusters • Pre-copy approach: VM continues to run • ‘lift’ domain on to shadow page tables • Bitmap of dirtied pages; scan; transmit dirtied • Atomic ‘zero bitmap & make PTEs read-only’ • Iterate until no forward progress, then stop VM and transfer remainder • Rewrite page tables for new MFNs; Restart • Migrate MAC or send unsolicited ARP-Reply • Downtime typically 10’s of milliseconds • (though very application dependent)

  24. Scalability • Scalability principally limited by Application resource requirements • several 10’s of VMs on server-class machines • Balloon driver used to control domain memory usage by returning pages to Xen • Normal OS paging mechanisms can deflate quiescent domains to <4MB • Xen per-guest memory usage <32KB • Additional multiplexing overhead negligible

  25. Scalability 1000 800 600 400 200 0 L X L X L X L X 2 4 8 16 Simultaneous SPEC WEB99 Instances on Linux (L) and Xen(X)

  26. Resource Differentation 2.0 1.5 Aggregate throughput relative to one instance 1.0 0.5 0.0 4 4 2 8 2 8 8(diff) 8(diff) OSDB-IR OSDB-OLTP Simultaneous OSDB-IR and OSDB-OLTP Instances on Xen

  27. On-Going Work • xend web control interface • Cluster management tools • Load balancing • SMP guest OSes (have SMP hosts already) • Support for Intel VT/LT x86 extensions • Will enable full virtualization • VM Checkpointing • Debugging and fault tolerance

  28. Conclusions • Xen is a complete and robust GPL VMM • Outstanding performance and scalability • Excellent resource control and protection • Late “RC” of 2.0 out now: • Linux 2.4.27 & 2.6.8.1, NetBSD 2.0 • CPU, memory, network isolation • Basic support for ballooning, migration http://xen.sf.net

More Related