1 / 47

Process Migration Checkpoint/Restart

Process Migration Checkpoint/Restart. ECI, July 2005. Process Migration. Process migration benefits: Tool for load balancing Data access locality Improved system administration Mobile computing. Process Migration Issues. Execution model: home, remote Migrating virtual memory

kiefer
Download Presentation

Process Migration Checkpoint/Restart

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Process MigrationCheckpoint/Restart ECI, July 2005

  2. Process Migration • Process migration benefits: • Tool for load balancing • Data access locality • Improved system administration • Mobile computing

  3. Process Migration Issues • Execution model: home, remote • Migrating virtual memory • Minimizing downtime • Cost of migration • Run time cost (home, remote) • Migration operation • Limitations of migration

  4. Checkpoint / Restart • Checkpoint/restart benefits: • Like migration plus … • Fault resilience • Fault recovery • High availability • Gang scheduling • Debugging, testing, developing • Security (honey-pot)

  5. Checkpoint/restart goals • Transparency • Support parallel programs • Multi-process • Multi-node • Security • Minimize required state • Minimize required storage

  6. CKPT: Application Level • Application level • Efficient • Non-preemptive • Lack of common API • Source code changes • Possible compiler support • Examples ?

  7. CKPT: Library Level • Library level • Typically use a signal handler (callback) • Common API • Restricts functionality (e.g., no IPC) • Relatively portable • Examples…

  8. CKPT: Library (contd) • Libckpt • Memory exclusion, incremental, forked • Modify source code, link statically • Condor • Support memory mapping, shared libraries • Relink to special library (needs object file) • Score, co-check • Parallel applications • Modify communication layer

  9. Implementation (contd) • Kernel level • Loadable kernel module vs. change kernel • Preemptive / cooperative • Access to entire process state • Complex, less portable • Examples: Sprite, Zap • Virtual machines • (soon)

  10. Multi-process Checkpoint • Global state • A set of states from all processes • Consistent global state • If the state of A reflects a message received from B, then the state of B reflects sending • If the state of A reflect a message sent to B but not yet received, it must be part of the channel state

  11. Consistent Global State

  12. Multi-process Checkpoint • Uncoordinated checkpoint • Inspect data to find recovery line • Processes are independent, efficient • Domino effect, much storage

  13. Multi-process Checkpoint • Coordinated checkpoint • Centrally managed • Blocking • All processes suspended • Flush communication channels • Non blocking • Delay in triggers may yield inconsistency

  14. Multi-process Checkpoint • Communication-induced • Piggyback process checkpoint status and requests on messages • May require enforcing global checkpoint • Unpredictable checkpoint times

  15. Multi-process Checkpoint • Summary:

  16. Virtual Machines “Any problem in computer science can be solved by another layer of indirection” ECI, July 2005

  17. What is a Virtual Machine ? • An indirection layer below the execution environment seen by applications and OS • Decouple architecture and user perceived behavior of SW and HW resources from their physical implementation • Provide a uniform view of the underlying resources • Multiplex multiple virtual systems on a single (physical) resource

  18. VM History • 1960’s – Hypervisors (mainframes) • Time-share expensive hardware • No change to legacy software • 1980-90’s – Obsolete • Proliferation of cheap hardware • Hardware support neglected • Later 1990’s – Reincarnation • For complex MPP lacking OS infrastructure • 2000 - Today: Renaissance • Consolidation, isolation, reliability

  19. VM Benefits • Performance • Server consolidation • Efficient HW utilization • Adaptive resource balancing • Checkpoint/restart and migration • Security • Simple (reduced complexity) • Encapsulation and isolation • Mediation

  20. VM benefits (contd) • Reliability • Redundancy through replication • Disaster recovery • Deployment testing • And… • Quality of service • Transparent (for legacy SW) • Enhanced interoperability • Development & testing

  21. Server utilization Cumulative usage of 28 servers: Memory • 45% of RAM not used 99.9% of time • 25% of RAM never used concurrently CPU • 85% of CPU not used 99.9% of time • 81% of CPU never used concurrently Disk • 68% of storage space never used

  22. Virtualization levels • HOST entity: encapsulates the guest • GUEST entity: managed by the host Application programs Libraries API Operating system ABI ISA Hardware

  23. Application Application VMM Process virtual machine OS Hardware Process & System VM Application Application OS OS VMM Virtual machine Hardware

  24. VM at different levels • HW level • VMware, Xen, Denali, Virtual PC, UML • OS level • Virtual Servers, BSD Jail, Zap • Programming language level • Java, .NET • Network • VLAN, VPN

  25. VM Taxonomy • Process VM - virtual platform that exists solely to support the process • Unix • Emulators (interpreters) • Dynamic binary translators • Optimize by block translation and caching • Java – “compile once run everywhere” • Intermediate machine code • Optimize by native compilation on-the-fly

  26. VM Taxonomy (contd) • System VM - complete persistent system environment providing access to virtual hardware • Classic - bare HW • Hosted VM • Easy install and maintenance • Leverage native services of underlying OS • Multiprocessor virtualization

  27. Hardware Virtualization • Challenges to build virtual machines • Performance isolation • Scheduling priority • Memory demand • Network traffic • Disk Access • Support for various OS platforms • Small performance overhead

  28. Lack of Hardware Support • Ring aliasing • Non-faulting access to privileged state • Does the guest see the right state ? • Address space compression • Where does the VMM reside ? • Impact on transitions • Traps, SYSENTER, SYSEXIT • Interrupts masking • Hidden state

  29. Now What ? • Hardware extensions • Change semantics to support VM • Intel, AMD • Software virtualization • Translate code to emulate desired behavior • VMware • Paravirtualization • Xen, Denali

  30. Hardware Extensions for VM • Root mode • Runs VMM • Like ring-0 before • Non-Root mode • Runs guest OS • Less privileged • Mask of events to trap

  31. VMware • Hardware virtualization • CPU, memory, I/O • Suspend/resume • Live migration Design goals: • Compatibility • Performance • Simplicity

  32. VMware: CPU Virtualization • CPU Virtualization • Execute guest on bare hardware while retaining control by the VMM • Traps privileged ops & emulates their action • Challenge: lack of HW support • POPF and read access to privileged state • Solution: fast binary translation • Only kernel mode code • Eliminate unnecessary traps

  33. VMware: Memory Virtualization • Memory virtualization • Shadow page tables • Challenges: • Inefficient page replacement • Oversized due to replication • Solutions: • Ballooning • Content based sharing

  34. VMware: I/O Virtualization • Challenge: wide variety of devices and interfaces • Solution: • Hosted architecture • Trap through the VMM • Export special devices

  35. Xen: Paravirtualization • Provide some exposure to the underlying hardware • Better performance • Must modify OS to adapt • No modifications to applications

  36. Xen (contd) • Downgrade privilege of guest OS • Guest registers syscall and page-fault handlers with Xen • Partial access to page tables • Fast handlers for most exceptions • Expose set of simple device abstractions

  37. Xen (contd) • The cost of porting an OS to Xen: • Privileged instructions • Page table access • Network driver • Block device driver • <2% of code-base

  38. Denali • Lightweight protection domains • Minimalistic method geared for performance • Changes: • Idle loops - avoid busy wait • Interrupt queueing - save context switch • Interrupt semantics – “just”/”recent” • No virtual memory (!) • No BIOS – no legacy “crap” • Generic I/O devices

  39. Virtual Machine Migration • Optimizations: • Reduce memory state before snapshot • ballooning • Reduce total cost by incremental updates • COW hierarchy • Reduce start-up time by paging on-demand • Reduce transfer time relying on common data • Use hash functions to identify common blocks

  40. Virtual Machine Migration • Minimizing down time • Reduce size of VM state • Pre-copy static parts (or..) • Demand-copy static parts • Hot-copy dynamic parts

  41. OS Virtualization • Confine applications in containers • Advantages: • Fine granularity • Low overhead • Easier maintenance • Challenges • Transparency • Correctness • Extend OS: • Modify kernel, loadable module, library

  42. Isolation – BSD Jail • Create an isolated existing environment via software means. • Uses chroot (private root per jail) • Processes in a jail are isolated from files, processes, or network services in other jails. • A jail can be restricted to a single IP address.

  43. Specialized Virtualization – Linux VServer • Hosting (consolidation) • Experimentation • Education (do you trust students … ?) • Personal security box • Manage several "versions“ • Applications • Virtual servers • Per user firewall • Fail over servers • Honey-pots

  44. Specialized Virtualization – Linux VServer • Isolation • Processes, file system, IPC, network, super user capabilities • Kernel patch • Add a “context” tag per process/resource • syscalls to handle contexts (irreversible) • Challenges • Capture all holes (indirect access !) • Efficient storage

  45. General Virtualization – Zap • Virtualization for isolation • POD – PrOcess Domain • Private namespace • Virtualization for migration • Decouple process from OS • Capture state and reconstruct state

  46. Zap – virtualization • Process environment • Interpose on system calls • File system • Rely on “chroot” environment • Network • Per protocol methods • Challenges • Race conditions (smp) • Life-span of objects • Fast translation

  47. Zap – Migration • Checkpoint – outside process context • Capture process tree • Capture pod state • Capture per-process state • Restart – inside process context • Restore process tree • Restore processes • Example issues • Sharing • Deleted files

More Related