Memory Resource Management in VMware ESX Server

1. Memory Resource Management in VMware ESX Server 72130307 김정수 rememberkarl@gmail.com

2. Contents 1. What is ESX Server? 2. Memory Virtualization 3. Reclamation Mechanisms 4. Sharing Memory 5. Shares vs. Working Sets 6. References

3. 2 5 1 3 4 6 1. What is the ESX Server? Previously Virtual Machine System (VM/370, Disco etc.) VM VM VM VM Operating System Virtual Machine Monitor Hardware resources

4. 2 5 1 3 4 6 1. What is the ESX Server? VM VM VM VM ESX Server Direct Access Hardware resources - Benefits Higher I/O Performance Complete control over resource management

5. 2 5 1 3 4 6 2. Memory Virtualization Virtual Machine Real Page Table MMU Shadow PageTable (RDONLY) Kernel code & data.. stack PFN PFN data Trap VMM code Virtual Machine Monitor Virtual Physical Memory Physical Memory

6. 2 5 1 3 4 6 2. Memory Virtualization ESX Server - Server can monitor or interpose on guest memory accesses. - This can refer to memory without additional overhead.

7. 2 5 1 3 4 6 3. Reclamation Mechanisms - Overcommitment: The total size configured for all running virtual machines exceeds the total amount of actual machine memory. - Page Replacement : 1) It is just for moving some VM “physical” pages to a swap area on disk. 2) The virtual machine system must choose not only the VM from which to revoke memory, but also which of its particular pages to reclaim. 3) It causes a double paging problem. 4) When the meta-level policy is able to select the same page that the native guest OS policy would choose.

8. 2 5 1 3 4 6 3. Reclamation Mechanisms - Ballooning 1) Server can reclaim memory on guest OS whenever server wants. 2) There is no interface within the guest and use a private channel for its communication. 3) It supports almost operating systems with ballooning driver. (Linux, FressBSD, Windows)

9. 2 5 1 3 4 6 3. Reclamation Mechanisms It may be uninstalled, disabled explicitly, unavailable while a guest OS is booting,ortemporarily unable to reclaim memory quickly enough to satisfy current system demands. And reasonable balloon sizes may be imposed by various guest OS limitations. Is the ballooning perfect solution?

10. 2 5 1 3 4 6 4. Sharing Memory - Transparent Page Sharing : Disco introduced for eliminating redundant copies of pages.

11. 2 5 1 3 4 6 4. Sharing Memory - Content-Based Page Sharing 1) It eliminates the need to modify, hook, or even understand guest OS code. 2) It can identify more opportunities for sharing; by definition, all potentially shareable pages can be identified by their contents.

12. 2 5 1 3 4 6 4. Sharing Memory - Implementations (b) Real-World Page Sharing - By page sharing, ESX Server only uses about 67% of memory with 10 VMs. (a) Page-Sharing Performance

13. 2 5 1 3 4 6 5. Shares vs. Working Sets - Shared-based Allocation 1) For proportional-share allocation of space-shared resources 2) Shares-per-page = price 3) Revocation reallocates memory away from clients paying a lower price to those willing to pay a higher price. - Reclaiming idle memory 1) Previous attempts to cross-apply techniques from proportional-share CPU resource management to compensate for idleness have not been successful. 2) Idle memory tax : To charge a client more for an idle page than for on it is actively using. 3) The ESX Server idle memory tax rate is a configurable parameter that defaults to 75%.

14. 2 5 1 3 4 6 5. Shares vs. Working Sets - Experimental Results - The statistical estimate of active memory usage responds quickly as more memory is touched, tracking the fast moving average, and more slowly as less memory is touched, tracking the slow moving average. - “zero page thread” that runs only when no other threads are runnable. - Results that demonstrate the effectiveness of imposing a tax on idle memory.

15. 2 5 1 3 4 6 6. References [1] EdouardBugnion, Scott Devine, KinshukGovil, and Mendel Rosenblum. “Disco: Running Commodity Operating Systems on Scalable Multiprocessors,” ACMTransactions on Computer Systems, 15(4), November 1997.

16. Thank you