Virtualizing Operating Systems

1. Virtualizing Operating Systems Dr. Dorgham Sisalem

2. Agenda • What is meant with Virtualization? • Why bother? • Terminology • Classical approach? • Why do not we use it? • Virtualization methods • Full Virtualization • Paravirtualization • Hardware-Assisted Virtualization • Summary • Slides under http://www.iptel.org/~dor/papers/Sisalem_VM.ppt 2|

3. Classical Computer System User Process User Process User Process User applications: Compiled only for certain OS on certain ISA Each process believes to have access to all HW resources Operating System: Mediator between user and machine Distribute resources among processes Access IO on behalf of processes Only one OS at a time Compiled only for a certain ISA Operating System Instruction Set Architecture (ISA) Physical hardware: CPU, memory, IO devices Machine

4. Classical Virtual System UP UP UP UP UP UP UP UP UP Guest: Assumes certain ISA (and possibly OS) Each guest believes to have access to all HW resources and to be using its native ISA Guest OS Guest OS Guest OS ISA ISA ISA VMM: Hypervisor Mediator between guest and host Distribute resources among guests Access IO on behalf of guests Works on a certain ISA Virtual Machine Monitor (VMM) Instruction Set Architecture (ISA) Host Platform Host: Physical hardware: CPU, memory, IO devices Possibly operating system

5. What is a Virtual Machine Monitor? • Classic Definition (Popek and Goldberg ’74) • The VMM provides an environment for programs which is essentially identical with the original machine • Programs run in the environment show at worst only minor decrease in speed • The VMM is in control of system resources

6. Virtualization Benefits • Isolation • Fault isolation • Performance isolation • Portability • Independent of physical hardware • Enables migration of live, running VMs • Efficiency • Idle resources used by multiple users and services • Improved scalability • Add more resources for a customer

7. A short history of virtualization Since 1998: Save costs, clouds Virtualization to save costs by using the same system for multiple users and OS Cheap Hardware High performance Vmware, Xen, Parallels, Microsoft … offer virtualization solutions Seventies: Save costs Expensive Hardware Support time-sharing Multiple versions of OS IBM implements virtualization into its mainframes (VM/370) Eighties, nineties: Virtualization considered exotic technology Moderately expensive Hardware Moderate performance Support windows over mac or old games

8. What‘s in it for the developer? • Portable software must be tested on different platforms • The Different platforms will remain idle most of the time • Reduce costs, management and space needs BSD Linux Unix BSD Linux Unix VMM 8|

9. What‘s in it for hosting provider? • Data centers with no Virtualization • Multiple customer applications on the same OS • No isolation • Possible security problems • Server/customer • Idle resources • With Virtualization • Each customer on a virtual machine • Reduce costs, management and space needs Customer1 Customer 2 Customer 3 Cust 1 Cust 2 Cust 3 VMM 9|

10. What‘s in it for users? • Heterogeneous applications on the user’s host • Windows application on mac • Old games • … • Offer the user a wider range of applications Visio (only for windows= Keynote (only for mac OS) Atari Visio Keynote Atari Host 10|

11. VM Classification VM Process System Emulation High Level Language VM (Java VM) Hosted Type 2 Bare HW (Type 1) 11|

12. Process vs. System VM User process (UP) Windows application Linux application (ISA2) UP UP UP UP UP UP UP UP UP Win XP Win Vista Linux Runtime Runtime Operating System (Linux) Virtual Machine Monitor Instruction Set Architecture (ISA) Instruction Set Architecture (ISA) Machine Host Platform Process VM: Enable a single process to run on the host System VM: Enable a complete system (OS + applications) to run on the host 12|

13. Process VM: Emulation • Adapt from one OS to another • Adapt from one ISA to another • Virtualize the Application Binary Interface (ABI) • User ISA • System calls to OS • Virtualization software intercepts the application’s system calls and translates them into the hosts system calls • If guest and host ISA are different then translate these as well • Examples: Wine Guest Virtualizing Software ABI Operating System Instruction Set Architecture (ISA) Machine 13|

14. System VM: Hosted vs. Bare HW VMM UP UP UP UP UP UP UP UP UP UP UP UP UP UP • Hosted VM • VMM runs as a user process on top of the host OS • Easy to install • An additional application • VMM can use IP device drivers of host OS • VMM scheduled as user process • No control on execution and resources • Less efficient • Examples: VM Workstation. Parallels .. Win XP Win Vista Win XP Linux Linux User Process (UP) Virtual Machine Monitor Virtual Machine Monitor Machine Host OS Machine • Bare HW VM • VMM runs directly on the hardware • Need to install the machine from scratch • VMM offers the IP devices to the guests • VMM controls which guest gets which share and when • Examples: VM ESX, Qemu, Xen 14|

15. Virtualization: What are the Problems? • Isolation: Make sure that one guest does not harm another • Only VMM can use priviliged instructions • Memory management: Each guest believes it has access to physical memory • Map memory locations of guest to actual machine memory • IO: All guests want access to disc, network, USB .... • Schedule between guests and prevent one guest from disturbing others • Multiple guests share the same NIC but might want to have different IP addresses 15|

16. Computer Architecture: CPU User ISA: Non-Priviliged CPU calculations Memory read-write in user memory space Non-Privileged Instruction: User Privileged Instruction: Kernel Kernel ISA: Priviliged IO access Manage user memory space Create/destroy processes Context switch Application Run calculation on CPU Raise system call for IO access Kernel takes over and issues privileged instruction 2 OS 3 1 ABI Instruction Set Architecture (ISA) Machine 16|

17. Classical Virtualization Approach: CPU User: Non-Privileged CPU calculations Memory read-write in user memory space Even less Privileged Instruction: User Non-Privileged Instructions: Kernel Kernel: Non-Privileged IO access Manage user memory space Create/destroy processes Context switch Privileged Instructions: VMM VMM: Privileged IO access Manage Guest memory space Create/destroy guests Context switch between guests Need at lest three levels of privileges 17|

18. Classical Virtualization Approach: CPU • Run calculation on CPU • Raise system call for IO access • Kernel takes over and issues privileged instruction but in non-privileged mode • Command will fail and generate a trap • The VMM catches the trap and • Checks if the guest is allowed to do the action • Issues the command on behalf of the guest Application 2 Guest OS 3 ✗ 1 4 ABI ISA VMM 5 ISA Machine 18|

19. Computer Architecture: Memory Process 1 Process 2 Logical page numbers (LPN): Process believes to have access to all memory LPN Physical page numbers (PPN): OS maps LPN to a physical location PPN 19|

20. Classical Virtualization Approach: Memory Virtual Machine 1 Virtual Machine 2 Logical page numbers (LPN) Process 1 Process 2 Process 2 Process 1 • Physical page numbers (PPN): • Each guest believes to have • access to the entire machine • physical memory • No longer references physical memory PPN PPN Shadow pages • Machine page numbers (MPN): • Map physical pages to the machine • memory VMM MPN • Shadow page numbers: • Map logical pages to the machine memory • Reduces the effort for a two step lookup • Increases the complexity as the pages must be kept in sync with the guests 20|

21. Virtualizing X86 Ring3: User • X86 use the IA32 Instruction set • Offers four levels of priviliges • Is not virtualizable using the classical approach • Some proviliged instructione when run in non-proviliged mode fail silently • No trap! Ring2 Ring1: Guest OS Ring0: VMM 21|

22. Full Virtualization: Processor • Emulate x86 environment to the guest OS • Interpretation • Fetch one instruction at a time • Decode the instruction: get the registers and memory • Execute an Interpreter routine that provides the same functionality • Highly inefficient • 1 instruction in OS can cause hundreds of instructions in VMM • Sometimes used for process vitualization • Binary Translation • Translate source binary program to target binary before/during execution • Go through the binary code and replace problematic instructions • Generate compiled code • Higher startup costs: VMM need to understand what is going on • Higher performance: number of additional instructions in the range of tens • Optimization: • Optimize frequently used parts • Cache compiled segments 22|

23. Full Virtualitzation: Binary Translation Introduction to Virtual Machines, Carl Waldspurger Guest Code Translation Cache vEPC mov ebx, eax mov ebx, eax start cli mov [VIF], 0 and ebx, ~0xfff and ebx, ~0xfff mov ebx, cr3 mov [CO_ARG], ebx sti call HANDLE_CR3 ret mov [VIF], 1 test [INT_PEND], 1 jne call HANDLE_INTS jmp HANDLE_RET

24. Full Virtualization: Memory Management • Use shadow tables for memory management • Block memory segments of Guests • Guest access to memory segments results in a trap • VMM updates shadow tables • VMM needs to keep shadow and LPM in sync 24|

25. Full Virtualization: IO Devices OS process • IO access • Shared devices: Network card • Guest access to a virtual device is intercepted by the VMM • VMM conducts the actual access • Data received on the device cause an interrupt at the guest OS • VMM needs to implement the device drives for all kinds of devices • Dedicated device: display, keyboard … • Guest Bypasses VM: No need to virtualize the device • Not really that simple as some instance needs to translate between guest and IO memory • IO devices have their own memory • IO devices can access machine memory directly Driver Device OS process Driver Device Virtual device Driver Device 25|

26. Paravirtualization • VMM offers the Guests a modified ISA • Example: XEN • Guest operating system is modified to use the VMM • In Full Virtualization problematic parts are fixed by VMM • In Paravirtualization problematic parts are fixed by developer of OS • Use Hypercalls instead of the native OS calls • Memory management • Guest allocates the pages • Read access is still directly without VMM involvement • Updates to the pages afterwards is only through hypercalls • VMM validates the Guest’s write/delete instructions • Guest can apply changes in batches • Mapping from logical to physical to machine addresses is the responsibility of the Guest 26|

27. Paravirtualization: IO Access • IO access • Add Paravirtual device drivers to the Guest • VMM and Guest domain share an IO interface • XEN uses a circular buffer • After placing one or more requests in the buffer the Guest informs the VMM through a hyper call about the data • No need for interrupts OS process Paravirtual Driver IO Interface Driver Device 27|

28. Paravirtualization • Advantages • Higher performance than Full virtualization • Guest still has access to some real resources (time and machine addresses) • Disadvantage • Need to modify the Guest OS • Annoying even if <1% • Changes are different for each OS • Paravirtual drivers must be implemented for each OS 28|

29. Hardware Assisted Virtualization Ring 0 : User • Add an additional privilige level • Guest OS runs in non-Root level • Instructions that required Emulation or Paravirtualization cause a switch from non-root to root mode • Hardware takes over the logical-2-physical-to-Machine mapping • For every PPN access the hardware checks also the MPN • No need for shadow tables • Better support for dedicated devices • Hardware takes over the mapping of logical address to device address • No need for virtual device drivers • No need for emulation or paravirtualization Ring 2: Ring 1 Ring 0 (non-Root): Guest OS Ring 0 (root): VMM 29|

30. Summary • What are the benefits? • Isolation, portability, scalability • What are the usage scenarios? • Development, data centers, heterogenous applications • What is the classical way of doing virtualization • Trap and emulate • Why doesn‘t the x86 allow for virtualization in the classical manner • Not all insructions of the IA32 cause a trap • What are the common virtualization techniques • Full virtualization: Uses Binary translation, hides the machine but is complex and can be slow • Paravirtualization: Uses Hypercalls, is faster and simpler than full virtualization but requires changes to the OS • Hardware assisted: Remove some of the limitations of Full Virtualization • Remember which? 30|

31. References • Virtual Machines: Versatile Platforms For Systems And Processes, James E. Smith, Ravi Nair, May 2005, ISBN 1-55860-910-5 • “The Architecture of Virtual Machines". Smith, J. E.; Nair, R.(2005) Computer (IEEE Computer Society) • “Understanding Full Virtualization, Paravirtualization, and Hardware Assist” • “A Comparison of Software and Hardware Techniques for x86 Virtualization”, K. Adams and O. Ageson. Proceedings of ASPLOS 2006, October 2006 • “Xen and the Art of Virtualization”, P. Barham et al. Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), October 2003 • "Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization". G. Neiger et al.; Intel Intel Technology Journal 10 (3); 2006 • “Intel Virtualization Technology for Direct IO”, D. Abramson et al. ; Intel Technology Journal 10 (3); 2006 31|

32. Dorgham Sisalem Director Strategic Architecture Mobile: +49 171 304 2053E-mail: dorgham.sisalem@tekelec.com Thank you !! Questions 32|