Framework for Benchmarking Disk I/O and Scalability in Virtual Machine Environment

1. A Framework To Benchmark Hard Disk I/O And Scalability In A Multiple Virtual Machine EnvironmentTravis FinchResearch Advisor: Dr. WeberMcNair Scholars ProgramAugust 13, 2007

2. Motivation- Moore’s Law- System performance to roughly double every 24 months- Software applications have not changed as drastically to take full advantage of these abundant resources - A primary goal should be to best utilize the available computing resources and reduce overall costs- How?

3. Virtualization Overview- Server virtualization executes multiple, separate instances of operating systems on virtual hardware - Virtual machine monitors (VMMs) accomplish this by multiplexing the physical hardware - Works the same way traditional computer systems work where only one program runs on the physical hardware a time- Virtualization also provides a way to separate logical server roles, can ease system administration and maintenance

4. Virtualization Overview- End-user expectations should remain realistic- A utility is needed that accurately measures the system performance and overhead requirements of virtualization- The majority of benchmarks focus on measuring traditional execution of software applications- A benchmark is needed to test the performance of multiple virtual machines running concurrently

5. Approach- This research introduces a framework for developing benchmark applications that test hardware components in a multiple virtual machine environment- What is a framework?- Message-Passing Interface (MPI) will be used as a synchronization mechanism that allows discrete virtual machines to communicate with each other

6. Related Work- A problem with performance analysis of virtualization software is that the cost of initial test environment setup can be extraordinarily high- VMmark: a benchmark that measures the capability of a virtual server environment consisting of many virtual machines with a diverse workload- Although VMmark provides accurate and consistent results over many runs, deployment of a test environment can take more than a week.

7. Design- Target component: hard disk drive (HDD)- The exact same HDD-intense instruction sequence will execute on each virtual node - Although MPI is traditionally used to execute applications on multiple high-performance physical machines, it provided the necessary routines to accomplish the needed tasks for the benchmark

8. Research Questions1.) Does the presented framework provide an adequate mechanism for virtual machine setup and synchronization?2.) Is the performance degradation of multiple, simultaneous disk accesses on virtual machines linear as more instances are created? 3.) What is a reasonable limit for the number of virtual machines executing simultaneously without causing severe performance penalties?

9. Experiments- Debian 4.0 was chosen as the OS – OpenVZ as the VMM- Without accounting for the time spent downloading packages, test environment setup time was less than an hour- Custom scripts are used to launch and destroy any number of virtual machine instances

14. Results- The OpenVZ VMM did a good job of not starving any VM from execution time - Bandwidth speeds between each node were never greater than 0.60 MB/sec- Averaged a difference of 0.18 MB/sec

15. Research QuestionsTo answer the three questions:- The framework provided an excellent mechanism for VM setup and synchronization- Results show that the decline in system performance is definitely not linear in nature as more VMs are launched- It is difficult to quantify an encompassing limit of virtual machines executing simultaneouslyWhy?

16. Future Work - Improving the HDD benchmark and expanding the code to stress other hardware components- The deployment process can be expanded and manipulated to fit other virtual machine monitors - Expand the framework to enable running commands, scripts, or tests not written in the C programming language