Computing Frontiers May 2005 J. E. Smith

1. Virtual Machines Supporting Changing Technology and New Applications Computing Frontiers May 2005 J. E. Smith

2. Introduction Why are virtual machines interesting? They involve computer architecture in a pure sense They allow transcending of interfaces (which often seem to be an obstacle to innovation) They enable innovation in flexible, adaptive software & hardware, security, network computing (and others) Virtualization technologies will be a key part of most future computer systems VMs (c) 2005, J. E. Smith

3. Outline • Virtualization • The Architecture of Virtual Machines • Emulation • Enhancing Security • The Grid • Portable Environments • Co-Designed VMs VMs (c) 2005, J. E. Smith

4. file file abstraction Abstraction • Computer systems are built on levels of abstraction • Higher level of abstraction hide details at lower levels • Example: files are an abstraction of a disk VMs (c) 2005, J. E. Smith

5. virtualization file file Virtualization • Similar to abstraction Except • Details not necessarily hidden • Construct Virtual Disks • As files on a larger disk • Map state • Implement functions • VMs: do the same thing with the whole “machine” VMs (c) 2005, J. E. Smith

6. The Family of Virtual Machines Including things not called “virtual machines” IA-32 EL HP Dynamo Transmeta Crusoe There are lots of “virtual machines” IBM VM/370 Java VMware products “The subjects of virtual machines and emulators have been treated as entirely separate. … they have much in common. Not only do the usual implementations have many shared characteristics, but this commonality extends to the theoretical concepts on which they are based” -- Efrem G. Wallach, 1973 VMs (c) 2005, J. E. Smith

7. “Machines” Application Programs • Different perspectives on what the Machine is: • OS developer • Compiler developer • Application programmer Instruction Set Architecture • ISA • Major division between hardware and software Libraries Operating System Execution Hardware Application Binary Interface • ABI • User ISA + OS calls Application Program Interface • API • User ISA + library calls Memory Translation System Interconnect (bus) I/O devices Main and Memory Networking VMs (c) 2005, J. E. Smith

8. System Virtual Machines guest guest guest guest guest guest • Provide a system environment • Constructed at ISA level • Persistent • Examples: IBM VM/360, VMware, Transmeta Crusoe process process process process process process Guest OS2 Guest OS VMM VMM HOST PLATFORM virtual network communication VMs (c) 2005, J. E. Smith

9. Process Virtual Machines • Constructed at ABI level • Runtime manages guest process • Guest processes may intermingle with host processes • Not persistent • As a practical matter, guest and host OSes are often the same • Dynamic optimizers are a special case • Examples: IA-32 EL, FX!32, Dynamo guest host process process runtime guest guest host process process process runtime runtime create HOST OS file sharing Disk network communication VMs (c) 2005, J. E. Smith

10. HLL Program HLL Program Compiler front-end Compiler Intermediate Code Portable Code Virtual ISA ( ) Compiler back-end VM loader Object Code ISA ( ) Virt. Mem. Image VM Interpreter/Translator Loader Memory Image Host Instructions HLL VM Traditional High Level Language Virtual Machines • Raise the “ABI” level of abstraction • User higher level virtual ISA • OS abstracted as standard libraries • A form of process VM VMs (c) 2005, J. E. Smith

11. The Virtual Machine Space Process VMs System VMs different different same ISA same ISA ISA ISA Classic Whole Dynamic Multi OS VMs System VMs Translators programmed Systems Hosted Co-Designed Dynamic HLL VMs VMs VMs Binary Optimizers VMs (c) 2005, J. E. Smith

12. Key Feature – State/Resource Mapping • VM SW can Re-map logical to physical state • Via pointers or copying • Registers to registers • Registers to memory • Memory to disk VMs (c) 2005, J. E. Smith

13. Key Feature – Emulation • Interpretation • Software loop decodes and dispatches each instruction • Binary translation and code caching • Translate blocks of instructions at a time • Hold translated blocks in code cache • With same-ISA scanning/patching is an alternative • Staged Emulation • Emulation techniques invoked in staged manner • Based on performance tradeoffs VMs (c) 2005, J. E. Smith

14. Code Caches • Contain • Basic blocks • Superblocks (one entrance, multiple exits) • Optimized Superblocks • A base technology for many VMs • Dynamic binary translators: Intel IA-32 EL, Compaq FX!32 • Dynamic binary optimizers: Dynamo family • Co-designed virtual machines: Transmeta, IBM DAISY • High performance Java virtual machines • System VMs with “inefficiently virtualizable” ISAs • “Sandboxing” secure VMs (x86 DynamoRIO) VMs (c) 2005, J. E. Smith

15. Code Caching with Chaining • Chaining of blocks in code cache minimizes VM overhead Code Cache Superblock Dispatch table lookup code Superblock Superblock Superblock VMs (c) 2005, J. E. Smith

16. Staged Emulation • Start interpreting • Profile to find “hot” code regions • An important part of many VM implementations • Translate, optimize & cache frequent code sequences Interpreter Binary Memory Profile Data runtime Code Cache Image Translator/ Optimizer VMs (c) 2005, J. E. Smith

17. Key Feature – VMM/Runtime Control • Interpretation • Fine grain control • Every dynamic instruction “inspected” before execution • Binary translation and code caching • Coarser grain control • Every static instruction inspected before execution • Jumps to VM SW can be inserted anywhere • Protection levels • Very coarse grain control • Every resource-related instruction trapped by protection system • Otherwise, use interpretation/translation techniques • Used in system VMs to manage resource mappings VMs (c) 2005, J. E. Smith

18. VMM Resource Control in System VMs Application system call/trap • Traps and interrupts (& sys calls) • Transfer to VMM • VMM determines appropriate Guest OS • VMM transfers to Guest OS • Guest OS “return” to user app. • Transfer to VMM • VMM bounces return back to Guest app. • Resource sensitive instructions • Trap to VMM • VMM checks correctness • VMM reads/modifies guest resource • Returns to Guest Guest OS privileged operation next instruction virtual vector location: VMM check privileges perform operation return vector location: VMs (c) 2005, J. E. Smith

19. VMM as a Smart Interconnect • Two modes: • Execution mode • VM mode • After it gains control • VM SW can manage resources via state mapping • VM SW can alter/enhance functions via emulation apps 2 OS 2 apps 2 apps 1 apps 1 apps 1 OS 2 OS 1 OS 1 OS 1 ISA 1 ISA 1 ISA 1 VMs (c) 2005, J. E. Smith

20. Security • Many security threats • Worms, viruses, Trojan horses, etc. • Typical attack – get access to privileged part of system • Often with little effort • Compromised passwords • “Easy” passwords • Mechanically repeated efforts • Exploit weakness in system software • Unchecked accesses to system data structures • Can get control in privileged state by causing overflows VMs (c) 2005, J. E. Smith

21. Buffer Overflow User Mode Supervisor Mode User invokes system program User Mode Supervisor Mode with faulty input that causes buffer overflow in stack User invokes Return address in stack system program clobbered due to with normal input overflow. Vulnerable system program peforms System program function and returns to performs function illegal address and returns to user User performs subsequent task System exception! (a) Normal Input (b) Faulty Input VMs (c) 2005, J. E. Smith

22. Malicious Input – Intrusion User Mode Supervisor Mode Malicious user invokes system program with tailored input that causes Return address in stack buffer overflow in stack changed due to overflow. Vulnerable system program peforms function and returns to user-specified address, e.g. address of shell program User gets full control of system through shell program running in supervisor mode VMs (c) 2005, J. E. Smith

23. Intrusion Detection Systems • Isolation is not an option • Increasing dependence on communication over networks • Language-level checking • Java, MSIL – range- and type-checking • Legacy applications and legacy style not protected • Need for Intrusion Detection Systems (IDS) • Depend on knowledge of potential attacks • Network-based Intrusion Detection Systems (NIDS) • Host-based Intrusion Detection Systems (HIDS) VMs (c) 2005, J. E. Smith

24. Host Intrusion Detection Systems • Directly examine activity on host • Knowledge of host operating system • Look for repeated attempts • To crack password • To access unauthorized files, etc. • HIDS has significantly better viewpoint compared to NIDS • But HIDS can be disabled by attack • Or can provide misleading information VMs (c) 2005, J. E. Smith

25. Monitoring and Recovering from Attacks • Importance of understanding attacks • To recover from an attack • To prevent future attacks • Logging • Save information about critical activity on system • Know the events that caused the failure • Save checkpoint of state of system • Reconstruct the attack from a known good state VMs (c) 2005, J. E. Smith

26. Virtual Machines as a Sandbox • Fault containment important feature of VMs • VM Isolation helps in close examination of attack • Clone system that has been attacked for later analysis • Use VM as a “honey-pot” • Permit attacks that can be monitored Production Virtual Machines VM1 VM2 VM3 VM4 Virtual Machine Monitor Hardware VMs (c) 2005, J. E. Smith

27. Virtual Machine for Monitoring • Livewire system (Stanford) • Separates IDS from VMM • IDS configures the VMM to monitor activity at more than the usual points • Signature of suspicious activity may be specified • After initialization, IDS enters the picture only in analyzing data from suspicious activity • Feedback – suggest new monitoring based on analysis • E.g. monitor system call activity after repeated login attempts • May need knowledge of OS to analyze data, e.g. crash dumps VMs (c) 2005, J. E. Smith

28. Livewire IDS IDS Policy Engine Guest Virtual Machine Policy Modules Guest Apps Policy Framework Config File Command Guest Query Response OS OS Interface Library Guest OS Metadata Callback Virtual Machine Monitor Hardware VMs (c) 2005, J. E. Smith

29. Policy Modules in Livewire • Polling modules • Lie detector module • VMM knows hardware state for each virtual machine • Lie detector compares this state to the state provided as feedback from intruder • User program integrity detector module • Compare signatures of memory pages with saved signatures • Signature detector module • Scan memory with signature of known viruses, Trojan horse programs, etc. • Event-driven modules • Memory access enforcer module • VMM intercepts attempts to change page access privileges VMs (c) 2005, J. E. Smith

30. Dynamic Binary Rewriting • Program shepherding • Control execution of program • Prevent program from being attacked • Prevent program from being launching point for attacks • RIO System (MIT) • Based on Dynamo binary optimization system • Target of every control transfer instruction verified • Not to unauthorized locations • Only to safe locations VMs (c) 2005, J. E. Smith

31. RIO Dynamic Binary Rewriting System Indirect Branch Lookup Routine Basic Block Cache Superblock Cache Application Mode RIO Mode Dispatch Routine START Basic Block Builder Superblock Selector • Two levels of translation • Quick translation (basic blocks) • High performance translation (superblocks) • Security Checks • All code inspected during translation • All control transfers are checked before caching/table placement • Code cache and map table are protected • Small performance loss VMs (c) 2005, J. E. Smith

32. Migration of Computing Environments • Identical environment at any work location • When moving from one location to another • E.g. Home to work and back • Effect similar to carrying hardware back and forth • Physical security has to be taken care of • Entire state of machine must be transported • State of processor resources • For OS as well as applications • Includes active code and data • Concept of a capsule • Compressed information about entire system • Can be transported from one location to another VMs (c) 2005, J. E. Smith

33. Virtual Computers • Encapsulation simplified through use of virtual machines • Encapsulation has the effect of checkpointing • Suspend operation on one platform and resume execution at exactly same point on another platform Traditional Data a a Migration t t a a Apps1 Apps2 D D OS1 OS2 Hardware 1 Hardware 2 Virtual Machine Virtual Machine Guest Apps VM Migration Guest Apps Guest OS Guest OS Virtual Machine Monitor 1 Virtual Machine Monitor 2 Hardware 1 Hardware 2 VMs (c) 2005, J. E. Smith

34. VMotion (VMware) • Migration of virtual machines in commercial environment • Load balancing • Security, e.g. quarantine attacked machine • Co-location • Fault-tolerance • Power management • Maintenance VC Client VC Client VC Client VC Client (User 1) (User 2) (User 3) (User 4) VC Management Server VC Database VCagent VCagent VCagent VM1 VM2 VM3 VM4 VM5 VM6 VM7 VM8 VM9 hostA hostB hostC Data SAN Store VMs (c) 2005, J. E. Smith

35. Migration Steps • Step 1: Ensure that VM is stable on current host • Step 2: Perform baseline copy • Copy of current memory state and data • Step 3: Suspend VM on current host • Step 4: Perform final copy • Send incremental capsule containing changes since baseline copy • Step 5: Activate VM on new host VMs (c) 2005, J. E. Smith

36. Grids: Virtual Organizations VMs (c) 2005, J. E. Smith

37. Comparison with Conventional VMs • Efficient utilization of resources • Similar in motivation to original system VMs • Sharing of resources • Grid concerned with sharing of content also • Not just sharing of resources • Distributed control • Grid has global scope • Users negotiate with each other to share and use resources • Heterogeneous nodes • Nodes in a grid may be different types of machines • Adaptation of applications • Applications may need to be adapted for the grid • Portability of applications • Conceptually similar to goals of HLL VMs VMs (c) 2005, J. E. Smith

38. Role of System VMs in a Grid • Grid has to manage and schedule resources • Like an operating system • However, grid has to deal with heterogeneity • Accounting, for example, is dependent on accounting policies of each grid participant • System VM-based approach • Treat a VM as the unit of transactions on a grid • Not tasks, or programs • ( Figuieredo and Fortes) VMs (c) 2005, J. E. Smith

39. Information Service Application Server F) (Front End V1 V2 V3 Vn Virtual Machines (Back End) The Internet X User Physical Server P I Image Server D Data Server System-VM Based Grid VMs (c) 2005, J. E. Smith

40. Advantages of SVM based Approach • User isolation • Protect user from host and other users • Protect host from users • Platform independence • User specifies type of machine, not actual machine • Task management and accounting • Simplifies allocation and accounting • Allocate based on compute requirements • Charge based on performance of VM • Portability • Allows applications to be written for execution on the widest range of platforms • Eases encapsulation and migration of jobs between nodes on grid; e.g. Java VMs can be migrated VMs (c) 2005, J. E. Smith

41. User Applications User Applications libs. libs. OS OS V-ISA ISA Software I-ISA Hardware Hardware Co-Designed Virtual Machines • Separate the hardware/software interface from the ISA level of abstraction • Restore the ISA to its “natural” place  as an Implementation ISA that reflects actual hardware • Support existing ISAs  as a Virtual ISA • Let processor designers use both hardware and software • A form of system VM VMs (c) 2005, J. E. Smith

42. Co-Designed VMs • Should be of interest to both architects and micro-architects • Offers opportunities for performance, power saving, fault tolerance and other implementation-dependent features • Allows transcending conventional ISAs • IBM Daisy and Transmeta Crusoe • Don’t confuse them with VLIW! “pioneers are the ones with arrows in their backs” VMs (c) 2005, J. E. Smith

43. Architecture Issues: Concealed Memory • VM software resides in memory concealed from all conventional software Code ICache Cache concealed memory Hierarchy VM Code Processor VM Data Core Source ISA Code DCache conventional Hierarchy Source ISA Data memory VMs (c) 2005, J. E. Smith

44. Another Way of Doing Things conventional Func. Unit Translation Func. Cache Processor Main Memory Unit Unit Hierarchy Pipeline (form uops) . .. Func. Unit Main Memory dynamic translation Software Func. Translator Unit . .. Processor Cache Code Cache Translation Hierarchy Pipeline Unit (form uops) Func. Unit VMs (c) 2005, J. E. Smith

45. Fused Instruction Set • Co-designed VM x86 implementation • Shorten and simplify pipeline front-end • Combine pairs of dependent instructions • For single “unit” for pipeline processing • Use VM software to • “Crack” x86 instructions into RISC-ops • Re-order RISC-ops • Reassemble into (new) fused pairs • Related: Pentium-M fuses in front-end • Using original x86 instructions • “Reduced Splitting” is more accurate description VMs (c) 2005, J. E. Smith

46. 100% 90% 80% 70% 60% 50% Percentage of Dynamic Instructions ALU 40% FP or NOPs 30% BR ST 20% LD 10% Fused 0% 175.vpr 176.gcc 181.mcf 252.eon 254.gap Average 164.gzip 300.twolf 186.crafty 256.bzip2 255.vortex 197.parser 253.perlbmk Fusing Profile • About 50% of operations are fused • Only 5-10% of non-fused are single-cycle ALU ops VMs (c) 2005, J. E. Smith

47. Performance VMs (c) 2005, J. E. Smith

48. Summary • Many types of VMs • But common implementation technologies • A smart interconnect component • Should be studied/taught as a discipline on its own • Alongside OS, Application SW, HW • Many avenues for research • Lots of applications • Architecture meta-issues – • What features of OS, Applications, HW are “VM friendly”? • E.g. Goldberg work in early 70s for system VMs • Primitives for supporting VMs VMs (c) 2005, J. E. Smith