Georgia Tech November 2006 J. E. Smith

1. Virtual Machines: Supporting Changing Technology and New Applications Georgia Tech November 2006 J. E. Smith

2. Introduction Why virtual machines? They allow transcending of standardized interfaces (which sometimes are an obstacle to innovation) They enable innovation in flexible, adaptive software & hardware, security, network computing (and others) They involve computer architecture in a pure sense Virtualization technologies will be a key part of most future computer systems VMs (c) 2006, J. E. Smith

3. Outline • Virtualization • Virtual Machine Architecture • Virtual Machine Implementation • Computer Architecture Applications • Co-Designed VMs • Private Virtual Machines VMs (c) 2006, J. E. Smith

4. file file abstraction Abstraction • Computer systems are built on levels of abstraction • Higher level of abstraction hide details at lower levels • Example: files are an abstraction of a disk VMs (c) 2006, J. E. Smith

5. virtualization file file Virtualization • Similar to abstraction Except • Same level of detail • Construct Virtual Disks • As files on a larger disk • Map state • Map operations • VMs: do the same thing with the whole “machine” • Key concepts: Map state; Map Operations VMs (c) 2006, J. E. Smith

6. The Family of Virtual Machines There are lots of “virtual machines” IBM VM/370 Java VMware products Including things not called “virtual machines” IA-32 EL HP Dynamo Transmeta Crusoe “The subjects of virtual machines and emulators have been treated as entirely separate. … they have much in common. Not only do the usual implementations have many shared characteristics, but this commonality extends to the theoretical concepts on which they are based” -- Efrem G. Wallach, 1973 VMs (c) 2006, J. E. Smith

7. “Machines” Application Programs • Different perspectives on what the Machine is: • OS developer • Compiler developer • Application programmer Instruction Set Architecture • ISA • Major division between hardware and software Libraries Operating System Execution Hardware Application Binary Interface • ABI • User ISA + OS calls Application Program Interface • API • User ISA + library calls Memory Translation System Interconnect (bus) I/O devices Main and Memory Networking VMs (c) 2006, J. E. Smith

8. System Virtual Machines guest guest guest guest guest guest • ISA level • Provide a system environment • VMM manages guest OS + apps • Persistent • Examples: IBM VM/360, VMware, Transmeta Crusoe process process process process process process Guest OS2 Guest OS VMM VMM HOST PLATFORM virtual network communication VMs (c) 2006, J. E. Smith

9. Process Virtual Machines • ABI level • Runtime manages guest process • Guest processes may intermingle with host processes • Not persistent • Guest and host OSes are often the same • Dynamic optimizers are a special case • Examples: IA-32 EL, FX!32, Dynamo guest host process process runtime guest guest host process process process runtime runtime create HOST OS file sharing Disk network communication VMs (c) 2006, J. E. Smith

10. HLL Program HLL Program Compiler front-end Compiler Intermediate Code Portable Code Virtual ISA ( ) Compiler back-end VM loader Object Code ISA ( ) Virt. Mem. Image VM Interpreter/Translator Loader Memory Image Host Instructions HLL VM Traditional High Level Language Virtual Machines • Constructed at API level • User higher level virtual ISA • OS abstracted as standard libraries • A form of process VM VMs (c) 2006, J. E. Smith

11. Virtual Machine Architectures Process VMs System VMs different different same ISA same ISA ISA ISA Classic Whole Dynamic Multi OS VMs System VMs Translators programmed Systems Hosted Co-Designed Dynamic HLL VMs VMs VMs Binary Optimizers VMs (c) 2006, J. E. Smith

12. VM Technology – State Mapping Host Register Host Registers Space • VM SW re-maps virtual state to real state • Recall virtual disk • Registers to registers • Registers to memory • Memory to memory • Memory to disk Guest Registers Runtime Data Runtime Code Host ABI Guest Data Address Space Guest Code VMs (c) 2006, J. E. Smith

13. VM Technology – Operation Mapping • VM SW re-maps operations on state • Instruction-level state changes • Emulation • Protected state changes • OS operations • Done under VMM control • Key concepts: Emulation and Control VMs (c) 2006, J. E. Smith

14. VM Technology – Emulation interpreter source code routines • Interpretation • Software loop decodes and dispatches each instruction "data" accesses dispatch loop VMs (c) 2006, J. E. Smith

15. VM Technology – Emulation binary translated target code source code • Binary translation and code caching • Translate blocks of instructions at a time • Hold translated blocks in code cache • This was a key enabler for VMware success binary translator VMs (c) 2006, J. E. Smith

16. Emulation Interpreter manager Binary Memory Code Cache Profile Data Image Translator/ Optimizer VM Technology – Emulation • Staged Emulation • Emulation techniques invoked in staged manner • Based on performance tradeoffs VMs (c) 2006, J. E. Smith

17. Code Caches • Contain • Basic blocks • Superblocks (one entrance, multiple exits) • Optimized Superblocks • Used in many VMs • Dynamic binary translators: Intel IA-32 EL, Compaq FX!32 • Dynamic binary optimizers: Dynamo family • Co-designed virtual machines: Transmeta, IBM DAISY • High performance Java virtual machines • System VMs with “inefficiently virtualizable” ISAs • “Sandboxing” secure VMs (x86 DynamoRIO) VMs (c) 2006, J. E. Smith

18. Code Caching with Chaining • Chaining of blocks in code cache minimizes VM overhead Code Cache Superblock Dispatch table lookup code Superblock Superblock Superblock VMs (c) 2006, J. E. Smith

19. VM Technology – Control • Interpretation • Fine grain control • Every dynamic instruction “inspected” before execution • Binary translation and code caching • Coarser grain control • Every static instruction inspected before execution • Jumps to VM SW can be inserted anywhere • Protection levels • Very coarse grain control • Every resource-related instruction trapped by protection system • Otherwise, use interpretation/translation techniques • Used in system VMs VMs (c) 2006, J. E. Smith

20. Resource Control in System VMs Application system call/trap • Traps and interrupts (& sys calls) • Transfer to VMM • VMM determines appropriate Guest OS • VMM transfers to Guest OS • Guest OS “return” to user app. • Transfer to VMM • VMM bounces return back to Guest app. • Resource sensitive instructions • Trap to VMM • VMM checks correctness • VMM reads/modifies guest resource • Returns to Guest Guest OS privileged operation next instruction virtual vector location: system return VMM check privileges perform operation return vector location: VMs (c) 2006, J. E. Smith

21. VMs and Computer Architecture • Use virtualization to give computer architects a layer of software • Beneath all conventional software • Maintains vision of hardware as seen by conventional software • Performance optimizations via Co-designed VMs • VM SW can alter/enhance architecture via emulation • Resource management – Private Virtual Machines • VM SW can manage microarchitecture resources VMs (c) 2006, J. E. Smith

22. User Applications User Applications libs. libs. OS OS V-ISA ISA Software I-ISA Hardware Hardware Co-Designed Virtual Machines • Separate the hardware/software interface from the ISA level of abstraction • Restore the ISA to its “natural” place  as an Implementation ISA that reflects actual hardware • Support existing ISAs  as a Virtual ISA • Let processor designers use both hardware and software • A form of system VM VMs (c) 2006, J. E. Smith

23. VM Technology -- Concealed Memory • VM software resides in memory concealed from all conventional software • This software is available to hardware designer Code ICache Cache concealed memory Hierarchy VM Code Processor VM Data Core Source ISA Code DCache conventional Hierarchy Source ISA Data memory VMs (c) 2006, J. E. Smith

24. Co-Designed VMs • Of interest to both architects and micro-architects • Offers opportunities for performance, power saving, fault tolerance and other implementation-dependent features • Allows transcending conventional ISAs • Don’t confuse them with VLIW! • Early examples: IBM Daisy and Transmeta Crusoe “pioneers are the ones with arrows in their backs” VMs (c) 2006, J. E. Smith

25. Another Way of Doing Things conventional Func. Unit Translation Func. Cache Processor Main Memory Unit Unit Hierarchy Pipeline (form uops) . .. Func. Unit Main Memory dynamic translation Software Func. Translator Unit . .. Processor Cache Code Cache Translation Hierarchy Pipeline Unit (form uops) Func. Unit VMs (c) 2006, J. E. Smith

26. Fused Microarchitecture • Fuse dependent pairs of micro-ops to macro-ops • Current Intel approach • Use co-designed SW to achieve wider-scale fusing • Process & execute fused macro-ops as single Instructions throughout the entire pipeline • Allows pipelined wake-up/select issue logic VMs (c) 2006, J. E. Smith

27. Fusible Instruction Set • RISC-ops with unique features: • Fuse bit per instruction fuses two dependent instructions • Dense instruction encoding, 16/32-bit ISA design • Special Features to Support the x86 ISA • Condition codes • Addressing modes • Aware of long immediate & displacement values Core 32 - bit instruction formats F 10 b opcode 21 bit Immediate / Displacement F bit immediate 10 b opcode 16 / Displacement 5 b Rds F 10 b opcode 11 b Immediate / Disp 5 b Rsrc 5 b Rds F 16 bit opcode 5 b Rsrc 5 b Rsrc 5 b Rds Add - on 16 - bit instruction formats for code density F 5 b op 10 b Immd / Disp F 5 b op 5 b Rsrc 5 b Rds F 5 b op 5 b Rsrc 5 b Rds Fusible ISA Instruction Formats VMs (c) 2006, J. E. Smith

28. 100% 90% 80% 70% 60% 50% Percentage of Dynamic Instructions ALU 40% FP or NOPs 30% BR ST 20% LD 10% Fused 0% 175.vpr 176.gcc 181.mcf 252.eon 254.gap Average 164.gzip 300.twolf 186.crafty 256.bzip2 255.vortex 197.parser 253.perlbmk Fusing Profile • About 50% of operations are fused • Only 5-10% of non-fused are single-cycle ALU ops VMs (c) 2006, J. E. Smith

29. Performance Base+ Code Cache + fusing + shorter pipe + 3-1 ALU 70 60 50 40 Nomarlized IPC speedup (%) 30 20 10 0 -10 175.vpr 252.eon 254.gap 176.gcc 181.mcf 164.gzip 300.twolf 256.bzip2 186.crafty Harmonic 255.vortex 197.parser 253.perlbmk VMs (c) 2006, J. E. Smith

30. Virtual Private Machines • Multi-core systems will have many hardware-level shared resources • Multi-threaded processors • Multi-level shared caches • Shared memory ports • Spares for fault tolerance • And a number of important implementation dependences • Non-uniform memory delays • Power optimization features • Fault tolerance features VMs (c) 2006, J. E. Smith

31. Virtual Private Machines • Co-design a MicroVisor to provide software with Virtual Private Machines • Insulates conventional software from complicated implementation-dependent features • Provides performance virtualization • Unlike classic VMs • Quality of Service (QoS) • Performance Isolation System System VM VM Conventional OS Functional Hypervisor (VMM) Virtualization VPM VPM Performance MicroVisor Virtualization Multi-Core Hardware VMs (c) 2006, J. E. Smith

32. ICache μV Code concealed memory Hierarchy μV Data Processor Core Code DCache conventional Hierarchy Data memory MicroVisor • Virtualizes performance, not functionality • Co-Designed software to support hardware resource management • Concealed/isolated from all conventional software • Much larger than microcode, does not consume processor chip real estate • Uses conventional instructions (extended) so there is more likelihood of some cross-system portability VMs (c) 2006, J. E. Smith

33. Processes OS Maps Real Processors MicroVisor Maps Physical Processors “Real-izing” Processors/Memory • Separate Real Processors from Physical Processors • OS assigns processes to Real Processors • MicroVisor maps real processors to physical processors • MicroVisor also maps real memory to physical memory • Extend to cache memories VMs (c) 2006, J. E. Smith

34. Virtual Private Machines • MicroVisor maps high level requirements to hardware configuration • Requires mechanisms to provide microarchitecture level QoS Main Memory Main Memory Memory Controller Mem Mem Mem Mem Controller Controller Controller Controller L2 Cache L2 Cache L2 Cache L2 Cache L2 Cache Interconnection Net L1 Cache L1 Cache L1 Cache L1 Cache L1 Cache L1 Cache Proc. 0 Proc. 1 Proc. 0 Proc. 0 Proc. 0 Proc. 0 Threads Threads Thread 0 Thread 0 Thread 0 Thread 0 0 & 1 2 & 3 VPM 0 VPM 1 VPM 2 VPM 3 VMs (c) 2006, J. E. Smith

35. Applications • Performance optimization • Deal with NUMA • Provide QoS and performance isolation in multi-threaded systems • Power management • Adjust resources to match power constraints • Requires inferring demand for resources • in contrast to conventional OS • Fault tolerance • Detected fault triggers MicroVisor • Diagnose, reconfigure, re-map memory/processors VMs (c) 2006, J. E. Smith

36. Summary • Many types of VMs • But common virtualization technologies • An important system component • Should be studied/taught as a discipline on its own • Alongside OS, Application SW, HW • Many avenues for computer architecture research • Co-designed VMs • Virtual Private Machines • Adaptive microarchitecture • Fault-tolerant implementations • Primitives for supporting efficient VMs VMs (c) 2006, J. E. Smith