Virtualization

1. Virtualization Enforced Modularity Principles of Computer System (2012 Fall)

2. Virtualization • Widely used term involving a layer of indirection top of some abstraction

3. Virtualization: Multiplexing • Examples: • Thread: 1 CPU looks like N CPUs • Virtual Memory: 1 memory looks like N memories • Virtual Circuit: 1 channel looks like N channels • Virtual Machine: 1 machine looks line N machines • Virtual Links – Bounded buffers

4. Virtualization: Aggregation • Examples: • RAID disks: N disks look like one disk • Channel Bounding: N channels look like one • Compute side: GPUs? ModuleA VirtualizationLayer ModuleA ModuleB ModB ModB ModB

5. Virtualization: Emulation • Examples: • RAM Disk: Make a RAM memory act like a very fast disk • Apple’s Rosetta Technology: Make a x86 CPU look like a Power PC CPU • Virtual Machines: Virtual card punches, virtual card readers, virtual networks

6. Virtualization for enforced modularity • Multiple isolated modules on the same computer • Use Operating System abstractions to enforce modularity • Major abstractions: • Memory • Virtual memory address space – OS gives modules their own memory name space • Communication • Bounded buffer virtual link – OS gives modules an inter-module communication channel for passing msgs • Interpreter • Thread – OS gives modules a slice of a CPU

7. Virtual memory

8. Operating System • Goals • Multiplexing • Protection • Cooperation • Portable • Performance • Technologies • Virtualization, e.g., memory and CPU • Abstraction, e.g., file, bounded buffer, window

9. Memory modularity enforcement • No enforcement in physical memory • Any module can overwrite any other • All modules share the same address space (naming context) • Bad things happen Physicalmemorynamespace 0x9000 0x7000 0x4000 0x3000 0x0000 OS gcc disk image access emacs

10. Virtual memory address spaces • Each module gets its own names • Addresses running from 0 to maximum address • Can’t even name memory of other modules or OS • Error handling • Handles all types of Operating System memory smash errors 2n 0 2n 0 ModuleB ModuleA OperatingSystem Module

11. Virtualizing Memory • Virtual addresses • Virtual address spaces • Virtual memory manager

12. Virtual Address • decouple modules with indirection • From a naming point of view • the virtual memory manager creates a name space of virtual addresses on top of a name space of physical addresses • The virtual memory manager’s naming scheme • translates virtual addresses into physical addresses • Managing physical memory transparently • Virtual memory transparently simulate a larger memory than the computer actually possesses

13. Virtual Address

14. Virtual Address • The cost of copying the content of a domain, • Complexity and performance • must manage both virtual addresses and a physical address space • must allocate and deallocate them (if the virtual address space isn’t large), • must set up translations between virtual and physical addresses • The translation happens on-the-fly • may slow down memory references.

15. Translating address using page map

16. Virtual Address Spaces • provides each application with the illusion • it has a complete address space to itself • The virtual memory manager • gives each virtual address space its own page map • id ←CREATE_ADDRESS_SPACE () • block ← ALLOCATE_BLOCK () • MAP (id, block, page_number, permission) • UNMAP (id, page_number) • FREE_BLOCK (block) • DELETE_ADDRESS_SPACE (id)

17. Kernel mode • How to protect page tables? • Add one bit to the processor to indicate the mode • change the value of domain registers only in kernel mode • generate an illegal instruction exception otherwise • change the mode bit only in kernel mode • Mode switch • Start in K • K->U: using iret • U->K: interrupt and system call

18. UNIX Abstractions • Why not using libraries?

19. Kernel and address spaces • Each address space include a mapping of the kernel into its address space • Only the user-mode bit must be changed when mode switches • The kernel sets the permissions for kernel pages to KERNEL-ONLY • Kernel has its own separate address space • inaccessible to user-level threads • extend the SVC instruction to switch the page map address register

20. Monolithic kernel • The kernel must be a trusted intermediary for the memory manager hardware, • Many designers also make the kernel the trusted intermediary for all other shared devices • the clock, display, disk • Modules that manage these devices must be part of the kernel program • the window manager, network manager , file manager • Monolithic kernel • most of the operating system runs in kernel mode

21. Microkernel • We would like to keep the kernel small • Reduce the # of bugs • Restrict errors in the module which generates the error • the file manager module error may overwrite kernel data structures unrelated to the file system • causing unrelated parts of the kernel to fail

22. Microkernel • System modules run in user mode in their own domain • Microkernel itself implements a minimal set of abstractions • Domains to contain modules • threads to run programs • virtual communication links

23. Microkernel VS. monolithic kernel • Few microkernel OS • Most widely-used operating systems have a mostly monolithic kernel. • the GNU/Linux operating system • the file and the network service run in kernel mode • the X Window System runs in user mode.

24. Microkernel VS. monolithic kernel • A critical service doesn’t matter much if it fails • Either in user mode or in kernel mode • The system is unusable. • Some services are shared among many modules • easier to implement these services as part of the kernel program, which is already shared among all modules • The performance of some services is critical • the overhead of SEND and RECEIVE supervisor calls may be too large

25. Microkernel VS. monolithic kernel • Monolithic systems can enjoy the ease of debugging of microkernel systems • good kernel debugging tools • It may be difficult to reorganize existing kernel programs • There is little incentive to change a kernel program that already works • If the system works and most of the errors have been eradicated • the debugging advantage of microkernel begins to evaporate • the cost of SEND and RECEIVE supervisor calls begins to dominate.

26. Microkernel VS. monolithic kernel • How to choose • a working system and a better designed, but new system • don’t switch over to the new system unless it is much better • The overhead of switching • learning the new design • re-engineering the old system to use the new design • rediscovering undocumented assumptions • discovering unrealized assumptions

27. Microkernel VS. monolithic kernel • the uncertainty of the gain of switching • The claims about the better design are speculative • There is little experimental evidence that • microkernel-based systems are more robust than existing monolithic kernels

28. Exo-kernel

29. virtual link: bounded buffer

30. Virtual communication links 2n 0 2n 0 ModuleA ModuleB OS

31. Enforce modularity for bounded buffer • Put the buffer in a shared kernel domain, the threads • cannot directly write the shared buffer in user mode • must transition to kernel mode to copy messages into the shared buffer • SEND and RECEIVE are supervisor calls • change to kernel mode • copy the message from/to the thread’s domain to/from the shared buffer • Programs running in kernel mode is written carefully • Transitions between user mode and kernel mode can be expensive

32. Virtual communication links • Bounded buffers interface: buffer ← ALLOCATE_BOUNDED_BUFFER(n) DEALLOCATED_BOUNDED_BUFFER(buffer) SEND(buffer, message) Message ← RECEIVE(buffer) • Producer and consumer problem • Sender must wait when buffer is full • Receiver must wait when buffer is empty • Sequence coordination

33. Bounded Buffer Send

34. Bounded Buffer Send/Receive

35. Bounded Buffer Send/Receive

36. Send/Receive ImplementationAssumptions • Single producer & Single consumer • Each on own CPU • in and out don’t overflow • read/write coherence • in and out before-or-after atomicity • The result of executing a statement becomes visible to other threads in program order • Compilers cannot optimize it

37. Multiple Senders • Multiple senders may send at the same time

38. Race Condition • Race condition • Timing dependent error involving shared state • Whether it happens depends on how threads scheduled

39. Race conditions ─ Hard to Control • Must make sure all possible schedules are safe • Number of possible schedules permutations is huge • Bad schedules that will and will not work sometimes • They are intermittent • Small timing changes between invocations might result in different behavior which can hide bug • Heisenbugs (Heisenberg)

40. RaceCondition if two writes p.in=p.in+1 in = 0 time Thread2 Thread1 mov in, %eax(0) add $0x1,%eax mov %eax, in (1) movin,%eax(0) add$0x1,%eax mov%eax,in(1) in = 1 • Anotherpossibleinterleaving: • Everythingworksfine • Debuggingcanbeextremelydifficult

41. Before-or-After Atomicity • Code assumes that when the producer does: p.in = p.in+ 1 • and the consumer reads in: while (p.in == p.out) do nothing // Spin • will receive the value of in before or after the assignment • Most machines have atomicity when the variable size is less than or equal to the memory width

42. Before-or-After Atomicity • volatile int in; • Works on 32 and 64 bit machines mov in, %eax add $0x1, %eax mov %eax, in

43. Before-or-After Atomicity • volatile long long int in; • Works only on 64bit machines, not 32bit mov in, %eax mov in+4, %edx add $0x1, %eax adc $0x0, %edx mov %eax, in mov %edx, in+4

44. Locks: Before-or-after atomicity • Widely used concept in systems • With a bunch of different names • Database community • Isolation and Isolated actions • Operating systems community • Mutual exclusion (mutex) and Critical sections • Computer architecture community • Atomicity and Atomic actions

45. Using Locks • Developer must figure out the possible race conditions and insert locks to prevent them • Puts a heavy load on the developer • Developer must place acquire/release of locks in the code • Nothing is automatic • Forget a place and you have race conditions • Forget to release a lock or try to acquire it twice, and you have likely have a deadlock

46. Send with Locking

47. Does this work?

48. Lock Granularity • Systems can be distinguished by number of locks and the amount of share data they protect • Developer must choose a locking scheme to provides the need amount of concurrency • Frequently concurrency leads to complexity

49. Lock Granularity • Course-grain locking – Few locks protecting more data + Simpler, easier to reason about - Less concurrency • Fine-grain locking – Many locks protecting smaller pieces of data + High amount of concurrency - More complex - Overhead for locks

50. Simple example of granularity • Allocate a single lock and acquire the lock every time you enter the subsystem and release it when you leave the subsystem • Advantage: Simple. As long as subsystem doesn’t call into itself recursively everything is fine • Disadvantage: No concurrency • Frequently done if a subsystem doesn’t need concurrency while other parts of the system does • Example: Many large software projects • Many modern operating systems (Example: Unix/Linux, NT)