470 likes | 515 Views
Explore the benefits, issues, and implementation of process migration, checkpoint/restart, and virtual machine technology. Learn about load balancing, data locality, system administration improvements, and mobile computing advantages. Understand the execution models, minimizing downtime, cost considerations, and runtime costs. Dive into checkpoint/restart benefits like fault resilience, recovery, high availability, gang scheduling, debugging, testing, and security aspects. Discover checkpoint/restart goals, transparency, support for parallel programs, and security measures. Different levels of checkpoint implementations and virtual machine taxonomy are also discussed. Uncover the history, benefits, and utilization of virtual machines for performance, server consolidation, security, reliability, and server utilization optimization.
E N D
Process MigrationCheckpoint/Restart ECI, July 2005
Process Migration • Process migration benefits: • Tool for load balancing • Data access locality • Improved system administration • Mobile computing
Process Migration Issues • Execution model: home, remote • Migrating virtual memory • Minimizing downtime • Cost of migration • Run time cost (home, remote) • Migration operation • Limitations of migration
Checkpoint / Restart • Checkpoint/restart benefits: • Like migration plus … • Fault resilience • Fault recovery • High availability • Gang scheduling • Debugging, testing, developing • Security (honey-pot)
Checkpoint/restart goals • Transparency • Support parallel programs • Multi-process • Multi-node • Security • Minimize required state • Minimize required storage
CKPT: Application Level • Application level • Efficient • Non-preemptive • Lack of common API • Source code changes • Possible compiler support • Examples ?
CKPT: Library Level • Library level • Typically use a signal handler (callback) • Common API • Restricts functionality (e.g., no IPC) • Relatively portable • Examples…
CKPT: Library (contd) • Libckpt • Memory exclusion, incremental, forked • Modify source code, link statically • Condor • Support memory mapping, shared libraries • Relink to special library (needs object file) • Score, co-check • Parallel applications • Modify communication layer
Implementation (contd) • Kernel level • Loadable kernel module vs. change kernel • Preemptive / cooperative • Access to entire process state • Complex, less portable • Examples: Sprite, Zap • Virtual machines • (soon)
Multi-process Checkpoint • Global state • A set of states from all processes • Consistent global state • If the state of A reflects a message received from B, then the state of B reflects sending • If the state of A reflect a message sent to B but not yet received, it must be part of the channel state
Multi-process Checkpoint • Uncoordinated checkpoint • Inspect data to find recovery line • Processes are independent, efficient • Domino effect, much storage
Multi-process Checkpoint • Coordinated checkpoint • Centrally managed • Blocking • All processes suspended • Flush communication channels • Non blocking • Delay in triggers may yield inconsistency
Multi-process Checkpoint • Communication-induced • Piggyback process checkpoint status and requests on messages • May require enforcing global checkpoint • Unpredictable checkpoint times
Multi-process Checkpoint • Summary:
Virtual Machines “Any problem in computer science can be solved by another layer of indirection” ECI, July 2005
What is a Virtual Machine ? • An indirection layer below the execution environment seen by applications and OS • Decouple architecture and user perceived behavior of SW and HW resources from their physical implementation • Provide a uniform view of the underlying resources • Multiplex multiple virtual systems on a single (physical) resource
VM History • 1960’s – Hypervisors (mainframes) • Time-share expensive hardware • No change to legacy software • 1980-90’s – Obsolete • Proliferation of cheap hardware • Hardware support neglected • Later 1990’s – Reincarnation • For complex MPP lacking OS infrastructure • 2000 - Today: Renaissance • Consolidation, isolation, reliability
VM Benefits • Performance • Server consolidation • Efficient HW utilization • Adaptive resource balancing • Checkpoint/restart and migration • Security • Simple (reduced complexity) • Encapsulation and isolation • Mediation
VM benefits (contd) • Reliability • Redundancy through replication • Disaster recovery • Deployment testing • And… • Quality of service • Transparent (for legacy SW) • Enhanced interoperability • Development & testing
Server utilization Cumulative usage of 28 servers: Memory • 45% of RAM not used 99.9% of time • 25% of RAM never used concurrently CPU • 85% of CPU not used 99.9% of time • 81% of CPU never used concurrently Disk • 68% of storage space never used
Virtualization levels • HOST entity: encapsulates the guest • GUEST entity: managed by the host Application programs Libraries API Operating system ABI ISA Hardware
Application Application VMM Process virtual machine OS Hardware Process & System VM Application Application OS OS VMM Virtual machine Hardware
VM at different levels • HW level • VMware, Xen, Denali, Virtual PC, UML • OS level • Virtual Servers, BSD Jail, Zap • Programming language level • Java, .NET • Network • VLAN, VPN
VM Taxonomy • Process VM - virtual platform that exists solely to support the process • Unix • Emulators (interpreters) • Dynamic binary translators • Optimize by block translation and caching • Java – “compile once run everywhere” • Intermediate machine code • Optimize by native compilation on-the-fly
VM Taxonomy (contd) • System VM - complete persistent system environment providing access to virtual hardware • Classic - bare HW • Hosted VM • Easy install and maintenance • Leverage native services of underlying OS • Multiprocessor virtualization
Hardware Virtualization • Challenges to build virtual machines • Performance isolation • Scheduling priority • Memory demand • Network traffic • Disk Access • Support for various OS platforms • Small performance overhead
Lack of Hardware Support • Ring aliasing • Non-faulting access to privileged state • Does the guest see the right state ? • Address space compression • Where does the VMM reside ? • Impact on transitions • Traps, SYSENTER, SYSEXIT • Interrupts masking • Hidden state
Now What ? • Hardware extensions • Change semantics to support VM • Intel, AMD • Software virtualization • Translate code to emulate desired behavior • VMware • Paravirtualization • Xen, Denali
Hardware Extensions for VM • Root mode • Runs VMM • Like ring-0 before • Non-Root mode • Runs guest OS • Less privileged • Mask of events to trap
VMware • Hardware virtualization • CPU, memory, I/O • Suspend/resume • Live migration Design goals: • Compatibility • Performance • Simplicity
VMware: CPU Virtualization • CPU Virtualization • Execute guest on bare hardware while retaining control by the VMM • Traps privileged ops & emulates their action • Challenge: lack of HW support • POPF and read access to privileged state • Solution: fast binary translation • Only kernel mode code • Eliminate unnecessary traps
VMware: Memory Virtualization • Memory virtualization • Shadow page tables • Challenges: • Inefficient page replacement • Oversized due to replication • Solutions: • Ballooning • Content based sharing
VMware: I/O Virtualization • Challenge: wide variety of devices and interfaces • Solution: • Hosted architecture • Trap through the VMM • Export special devices
Xen: Paravirtualization • Provide some exposure to the underlying hardware • Better performance • Must modify OS to adapt • No modifications to applications
Xen (contd) • Downgrade privilege of guest OS • Guest registers syscall and page-fault handlers with Xen • Partial access to page tables • Fast handlers for most exceptions • Expose set of simple device abstractions
Xen (contd) • The cost of porting an OS to Xen: • Privileged instructions • Page table access • Network driver • Block device driver • <2% of code-base
Denali • Lightweight protection domains • Minimalistic method geared for performance • Changes: • Idle loops - avoid busy wait • Interrupt queueing - save context switch • Interrupt semantics – “just”/”recent” • No virtual memory (!) • No BIOS – no legacy “crap” • Generic I/O devices
Virtual Machine Migration • Optimizations: • Reduce memory state before snapshot • ballooning • Reduce total cost by incremental updates • COW hierarchy • Reduce start-up time by paging on-demand • Reduce transfer time relying on common data • Use hash functions to identify common blocks
Virtual Machine Migration • Minimizing down time • Reduce size of VM state • Pre-copy static parts (or..) • Demand-copy static parts • Hot-copy dynamic parts
OS Virtualization • Confine applications in containers • Advantages: • Fine granularity • Low overhead • Easier maintenance • Challenges • Transparency • Correctness • Extend OS: • Modify kernel, loadable module, library
Isolation – BSD Jail • Create an isolated existing environment via software means. • Uses chroot (private root per jail) • Processes in a jail are isolated from files, processes, or network services in other jails. • A jail can be restricted to a single IP address.
Specialized Virtualization – Linux VServer • Hosting (consolidation) • Experimentation • Education (do you trust students … ?) • Personal security box • Manage several "versions“ • Applications • Virtual servers • Per user firewall • Fail over servers • Honey-pots
Specialized Virtualization – Linux VServer • Isolation • Processes, file system, IPC, network, super user capabilities • Kernel patch • Add a “context” tag per process/resource • syscalls to handle contexts (irreversible) • Challenges • Capture all holes (indirect access !) • Efficient storage
General Virtualization – Zap • Virtualization for isolation • POD – PrOcess Domain • Private namespace • Virtualization for migration • Decouple process from OS • Capture state and reconstruct state
Zap – virtualization • Process environment • Interpose on system calls • File system • Rely on “chroot” environment • Network • Per protocol methods • Challenges • Race conditions (smp) • Life-span of objects • Fast translation
Zap – Migration • Checkpoint – outside process context • Capture process tree • Capture pod state • Capture per-process state • Restart – inside process context • Restore process tree • Restore processes • Example issues • Sharing • Deleted files