Diego ongaro alan l cox and scott rixner rice university march 5 2008
Download
1 / 24

Scheduling I/O in Virtual Machine Monitors - PowerPoint PPT Presentation


  • 207 Views
  • Updated On :

Diego Ongaro , Alan L. Cox, and Scott Rixner Rice University March 5, 2008. Scheduling I/O in Virtual Machine Monitors. Virtualized Host. Guest Domain 1. Guest Domain 2. Guest Domain 3. Guest Domain 4. . . . Server Consolidation. Streaming Media Server.

Related searches for Scheduling I/O in Virtual Machine Monitors

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Scheduling I/O in Virtual Machine Monitors' - tahir


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Diego ongaro alan l cox and scott rixner rice university march 5 2008 l.jpg

Diego Ongaro, Alan L. Cox, and Scott Rixner

Rice University

March 5, 2008

Scheduling I/O in Virtual Machine Monitors

Scheduling I/O in Virtual Machine Monitors


Server consolidation l.jpg

Scheduling I/O in Virtual Machine Monitors

Virtualized Host

Guest Domain 1

Guest Domain 2

Guest Domain 3

Guest Domain 4

. . .

Server Consolidation

Streaming Media Server

  • Virtual machine monitors enable server consolidation

  • Consolidated domains have wildly different workloads

  • Do current solutions actually achieve fairness?

Voice-over-IP Server

consolidation

Application Server

Web Server

Database Server

. . .


Xen performance l.jpg

Scheduling I/O in Virtual Machine Monitors

Xen Performance

Ping additional idle guest (G8):

Ping latency to G8: 259.2ms

Ping latency to G8: 6.6ms


Xen architecture l.jpg
Xen Architecture

Guest

Domain

User

Software

Driver

Domain

Xen Control

Software

Guest

Domain

User

Software

Guest

Domain

User

Software

  • Hypervisor – thin virtualization layer

  • Driver Domain – hardware control

  • Guest domains given restricted access

Xen Hypervisor

Hardware

Scheduling I/O in Virtual Machine Monitors


Receiving a network packet l.jpg

Scheduling I/O in Virtual Machine Monitors

time

Receiving a Network Packet

Guest

Domain

User

Software

Driver

Domain

Xen Control

Software

Guest

Domain

User

Software

Guest

Domain

User

Software

Xen Hypervisor

Hardware

VirtualInterrupt

VirtualInterrupt

HardwareInterrupt

Scheduler

Scheduler

Driver Domain

Packet Arrives

Target Domain


Scheduling delays l.jpg

Scheduling I/O in Virtual Machine Monitors

Scheduling Delays

  • Scheduler has a large effect on latency

    • As packets arrive, two opportunities to run any number of domains, delaying I/O

    • Delay proportional to the number of runnable domains

time

Scheduler

Scheduler

Driver Domain

Packet Arrives

Target Domain

Other Domains

Other Domains

Virtual Interrupt

Virtual Interrupt

Hardware Interrupt


Scheduler fairness l.jpg

Scheduling I/O in Virtual Machine Monitors

Scheduler Fairness

  • Virtual machine monitors must fairly support both computation and I/O domains

    • Xen supports computation domains well, but has mixed results with I/O domains

  • The scheduler plays a significant role in I/O performance

    • Improvements for I/O are not obvious

    • Some of Xen’s current scheduler optimizations for I/O are beneficial, others are not


Outline l.jpg

Scheduling I/O in Virtual Machine Monitors

Outline

  • Xen's Credit scheduler

  • Scheduler “optimizations” for I/O

  • Conclusions


Xen s credit scheduler l.jpg

Scheduling I/O in Virtual Machine Monitors

Xen’s Credit Scheduler

  • Credits are assigned to each domain

    • Approximate the fraction of processor resources each domain will receive

    • Do not indicate when each domain will receive its fraction

  • Scheduler increments/decrements credits

    • Periodically deducts credits from running domain

    • Adds credits when majority of credits in the system have been consumed


Scheduler operation l.jpg

Scheduling I/O in Virtual Machine Monitors

U4

U3

U2

Scheduler Operation

  • Domain states

    • Under – domain has credits remaining

    • Over – domain is over its credit allowance

  • Domains are run in FIFO order by state

    • Over domains only run if no under domains

    • A domain may run for up to 30ms if it has enough credits

  • After running, return to the run queue by state

    • Behind all other domains in the same state

    • Regardless of runtime or remaining credits

Run Queue

Next domain

to run

O2

O1

U1

This approach is biased against I/O


Scheduler optimizations for i o l.jpg

Scheduling I/O in Virtual Machine Monitors

Scheduler “Optimizations” for I/O

  • I/O Domain Requirements

    • Low latency

    • High bandwidth

    • Independent of other domains’ workloads

  • “Optimizations”

    • Boosting idle domains

      • Included in Xen

    • Ordering the run queue by credits

      • Proposed for Xen

    • Tickling the scheduler

      • Included in Xen


Boosting idle domains l.jpg

Scheduling I/O in Virtual Machine Monitors

O2

O2

O1

O1

U4

U4

U3

U3

U2

U2

U1

U1

Boosting Idle Domains

  • Initially, scheduler had no special features for I/O domains

  • Additional scheduling state: boost

    • Higher priority than under

    • Used when an idle domain is sent a virtual interrupt

  • With boost

Next domain

to run

U5

I1

Next domain

to run

B1

I1


Impact of boost l.jpg

Scheduling I/O in Virtual Machine Monitors

Impact of Boost

  • Boost improves latency for I/O domains


Caveats with boost l.jpg

Scheduling I/O in Virtual Machine Monitors

O2

O2

O1

O1

U4

B4

B3

U3

U2

B2

B1

U1

Next domain

to run

vs.

Caveats with Boost

  • Ineffective for I/O domains under heavy loads

    • Must be idle to receive boost

    • Must block quickly to sustain boost

  • Effect is negated with many I/O domains


Ordering the run queue l.jpg

Scheduling I/O in Virtual Machine Monitors

Ordering the Run Queue

  • I/O domains tend to quickly block

  • Within each state, sorts domains by credits remaining

  • Short-running I/O domains are re-inserted near the head of the run queue


Impact of ordering the run queue l.jpg

Scheduling I/O in Virtual Machine Monitors

Impact of Ordering the Run Queue

  • Ordering the run queue can reduce latency for I/O domains under larger loads

  • Surprisingly, often complements boost


Tickling the scheduler l.jpg

Scheduling I/O in Virtual Machine Monitors

Tickling the Scheduler

  • Xen tickles the scheduler after each virtual interrupt

  • Tickling can reduce latency

    • Often runs virtual interrupt recipient immediately

time

Scheduler

Scheduler

Driver Domain

Packet Arrives

Target Domain

Other Domains

Other Domains

Virtual Interrupt

Virtual Interrupt

Hardware Interrupt


Tickling too soon l.jpg

Scheduling I/O in Virtual Machine Monitors

Tickling Too Soon?

  • Tickling as virtual interrupts are sent may be too soon

    • Scheduler is uninformed about upcoming virtual interrupts

    • Gives arbitrary priority to first domain(s) to be sent a virtual interrupt

  • Don’t tickle while sending virtual interrupts

    • All recipient domains are then runnable

    • Scheduler runs the domain with the highest priority



Transcending xen l.jpg

Scheduling I/O in Virtual Machine Monitors

Transcending Xen

  • Scheduling mixed domains

  • Native vs. VMM scheduling


Mixed domains l.jpg

Scheduling I/O in Virtual Machine Monitors

Mixed Domains

  • Mixed I/O- and compute- domains don’t underutilize resources

  • VMM scheduler can not schedule only the I/O process

  • Low latency for a mixed domain can not be guaranteed

Driver

Domain

Guest

Domain

Guest

Domain

Guest

Domain

I/O process

CPU process

CPU process

CPU process

Hypervisor

Scheduler


Native vs vmm scheduling l.jpg

Scheduling I/O in Virtual Machine Monitors

Native

Driver

Domain

Guest

Guest

Guest

I/O

CPU process

CPU

CPU

CPU

CPU process

I/O process

?

CPU process

Hypervisor

Scheduler

Scheduler

Native vs. VMM Scheduling

Native

VMM


Summary of results l.jpg

Scheduling I/O in Virtual Machine Monitors

Summary of Results

  • Boost reduces latency with few I/O domains

    • Does little harm in other cases

  • Ordering the run queue favors short-running I/O domains

    • Often complementary to boost

  • Tickling reduces latency in some cases

    • Often causes unfairness in bandwidth and latency

  • I/O-intensive applications need their own domains


Conclusions l.jpg

Scheduling I/O in Virtual Machine Monitors

Conclusions

  • Scheduling has a large impact on I/O fairness

    • Can cause exorbitant latency

    • Can cause bandwidth inequity

  • Xen’s Credit scheduler is not fair for I/O

    • Does well with compute domains

    • No notion of timeliness, which is needed for I/O

  • Further work needs to be done to solve the mixed domains problem