Robert bradford evangelos kotsovinos anja feldmann harald schi berg presented by kit cischke
1 / 24

Live Wide-Area Migration of Virtual Machines Including Local Persistent State - PowerPoint PPT Presentation

  • Uploaded on

Robert Bradford, Evangelos Kotsovinos, Anja Feldmann, Harald Schiöberg Presented by Kit Cischke. Live Wide-Area Migration of Virtual Machines Including Local Persistent State. Differences in Contributions. The first paper discussed transferring run-time memory of a VM in a LAN. Cool.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Live Wide-Area Migration of Virtual Machines Including Local Persistent State' - teneil

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Robert bradford evangelos kotsovinos anja feldmann harald schi berg presented by kit cischke l.jpg

Robert Bradford, Evangelos Kotsovinos, Anja Feldmann, Harald Schiöberg

Presented by Kit Cischke

Live Wide-Area Migration of Virtual Machines Including Local Persistent State

Differences in contributions l.jpg
Differences in Contributions Schiöberg

  • The first paper discussed transferring run-time memory of a VM in a LAN.

    • Cool.

  • This paper expands on that to transfer the VM’s image, its persistent state and on-going network connections over a WAN as well.

  • By combining pre-copying, write-throttling and a block-driver, we can achieve this.

Introduction l.jpg
Introduction Schiöberg

  • In this project, the authors want to extend live VM migration to include:

    • Persistent state (file systems used by the VM)

    • Open network connections

  • Why?

    • Many apps running on a VM need that storage, and NAS systems may not be available in the new location.

    • Moving across a WAN will almost certainly involve an IP change, and we don’t want to (overly) disrupt TCP connections.

  • Contribution

    • A system enables live migration of VMs that use local storage and open network connections without severely disrupting their live services.

Highlights l.jpg
Highlights Schiöberg

  • Some highlights of this work:

  • Built upon the Xen Live Migration facility as part of XenoServer.

  • Enables:

    • Live migration

    • Consistency

    • Minimal Service Disruption

    • Transparency

  • Utilizes:

    • Pre-copying, write-throttling and IP tunneling.

System design environment l.jpg
System Design - Environment Schiöberg

  • Both the source and destination run Xen, with the VM running XenLinux.

  • Uses blocktap to export block devices into the migrated VM.

  • Block devices are file-backed meaning the contents of the block device are stored in an ordinary file on the file system of the VM.

System design architecture l.jpg
System Design - Architecture Schiöberg

  • The initialization stage starts things off by prepping the migration.

  • The bulk transfer stage pre-copies the disk image of the VM to the destination while the VM continues to run.

  • Xen transfer is then initiated, which performs incremental migration, again without stopping the VM.

  • While the transfers are occurring, all disk writes are intercepted as deltas that will be forwarded to the destination.

    • Deltas include the data written, the location written and the size of the data.

    • The deltas are recorded into a queue that will be transferred later.

  • If write activity is too high and too many deltas are being generated, write-throttling is engaged to slow down the VM.

  • In parallel with Xen transfer, the deltas are applied to the destination VM.

  • At some point, the source VM is paused, the destination is started and a temporary network redirect is created to handle the potential IP changes.

Implementation initialization l.jpg
Implementation - Initialization Schiöberg

  • Authentication, authorization and access control are handled by XenoServer.

  • The migration client forks, creating a listener process that signals the block-driver to enter record mode.

    • In record mode, the driver copies the writes to the listener process, which transfers them to the destination.

  • The other half of the migration client begins the bulk transfer.

  • At the destination, there is also a fork in the daemon. One receives the bulk transfer, the other receives the deltas.

Implementation bulk xfer l.jpg
Implementation – Bulk Xfer Schiöberg

  • The VM’s disk image is transferred from the source to the destination.

  • XenoServers platform supports copy-on-write along with immutable template disk images, so we can just transfer the changes, rather than the whole image.

Implentation xen migration l.jpg
Implentation – Xen Migration Schiöberg

  • The system here relies on the built-in migration mechanism of Xen.

  • Xen logs dirty memory pages and copies them to the destination without stopping the source VM.

  • Eventually, we will have to pause the source and copy the remaining memory pages.

  • Then we start up the migrated VM.

Implementation intercepting l.jpg
Implementation - Intercepting Schiöberg

  • The blkfront device driver communicates with the dedicated storage VM via ring buffer.

  • The blktap framework intercepts requests, but does it in user space.

  • Once a disk request makes it to the backend, it is both committed to the disk and sent to the migration client.

  • The client then packages the write up as a delta.

Implementation application l.jpg
Implementation - Application Schiöberg

  • After the bulk transfer, and in parallel with the Xen transfer, the deltas are transferred and applied to the migrated VM by the migration daemon in the storage VM at the destination.

  • If the delta queue becomes empty and the Xen migration is finished, I/O requests are put on hold until the application of the current crop of deltas is finished.

  • The authors found that delta application was normally finished before Xen migration, adding zero time to the overall migration.

Implementation write throttling l.jpg
Implementation – Write Throttling Schiöberg

  • If the VM attempts to complete more writes than a given threshold value, future write attempts are delayed by the block driver.

  • This process repeats, with the delay and threshold doubling each time.

  • Experimentally, 16384 is a suitable threshold with a delay of 2048 μs also being good.

  • Enforcement is separated from policy for extensibility.

Implementation wan redirection l.jpg
Implementation – WAN Redirection Schiöberg

  • If the IP of the VM changes, we use IP tunneling and Dynamic DNS to prevent dropped network connections.

  • Just before the source VM is paused, an IP tunnel is created from the source to destination using iproute2.

  • Once the destination VM is capable of responding to requests at its new IP, Dynamic DNS forwards the requests to the new IP.

  • Packets that arrive during the final stage of migration are simply dropped.

  • Once no connections exist that use the old IP, the tunnel is torn down.

  • Practically, this works because:

    • The source server only needs to cooperate for a short time, most network connections are short-lived and if nothing else, it’s no worse than what you get if the VM doesn’t even try.

Evaluation metrics l.jpg
Evaluation - Metrics Schiöberg

  • Want to evaluate the disruption of the system, as perceived by users.

    • Spoiler: Results look good.

  • Want to show the system handles diabolical workloads, defined in this paper as being heavy disk accessors, rather than heavy memory accessors.

  • Downtime: Time between pausing the VM on the source and resuming on the destination.

  • Disruption Time: Time during which clients observe a reduction in service responsiveness.

  • Additional Disruption: Difference between disruption time and downtime.

  • Migration time: Time from migration request to running VM at destination.

  • Number of Deltas and Delta Rate: How many file system changes and how often they occur.

Eval workload overview l.jpg
Eval – Workload Overview Schiöberg

  • Web server serving static content, serving dynamic web application and video streaming.

  • Chosen for realistic usage scenarios and because they neatly trifurcate the spectrum of disk I/O patterns:

    • Dynamic app generates lots of bursty writes

    • Static workload generates a medium amount of constant writes

    • Streaming video causes few writes, but is very sensitive to disruption.

Eval experimental setup l.jpg
Eval – Experimental Setup Schiöberg

  • Three hosts

    • Dual Xeon 3.2 GHz, 4 GB DDR RAM, mirrored RAID array of U320 SCSI disks.

  • The migrated VM was provided with 512 MB RAM and a single CPU.

  • All hosts were connected by a 100 Mbps switched Ethernet networks.

  • The migrated VM was running Debian on a 1GB ext3 disk.

  • Host C is the client.

  • To emulate WAN transfers, traffic shaping was used to limit the bandwidth to 5Mbps with 100 ms of latency.

    • Representative of host in London and U.S. east coast.

Results lan migration l.jpg
Results – LAN Migration Schiöberg

  • Measured disruption is 1.04 seconds and “practically unnoticeable by a human user.”

  • Few deltas by the log files for web server.

  • If you’re shooting for a “5 9’s” uptime, you still get 289 migrations a year.

Results lan migration18 l.jpg
Results – LAN Migration Schiöberg

  • phpBB with a randomly posting script.

  • Disruption is 3.09 seconds due to more deltas.

  • HTTP throughput is almost not affected and total migration time is shorter.

  • 5 9’s uptime still lets us migrate 98 migrations.

Results lan migration19 l.jpg
Results – LAN Migration Schiöberg

  • Streamed a large video file, viewed by a human on host C.

  • Disruption is 3.04 seconds and alleviated by the buffer of the video player.

  • No packets are lost, but there is lots of retransmission.

Comparison to freeze and copy l.jpg
Comparison to Freeze-and-Copy Schiöberg

  • Clearly, freeze and copy provides a much worse disruption than the live migration.

Results wan migration l.jpg
Results – WAN Migration Schiöberg

Longer migration time leads to more deltas.

The tunneling let the connections persist.

68 total seconds of disruption, which is a lot, but much less than freeze-and-copy.

Results diabolical workload l.jpg
Results – Diabolical Workload Schiöberg

Ran the Bonnie benchmark as a diabolical process, generating lots of disk writes. Needed to throttle twice. Initially, the bulk transfer is severely impeded and fixed. The overall migration takes 811 s, and without throttling, the transfer would have taken 3 days.

Results i o overhead l.jpg
Results – I/O Overhead Schiöberg

  • The overhead of intercepting deltas is pretty low and only noticeable during the migration.

Conclusions l.jpg
Conclusions Schiöberg

  • Demonstrated a VM migration scheme that includes persistent state, maintains open network connections and therefore works over a WAN without major disruption.

  • It can handle high I/O workloads too.

  • Works much better than freeze-and-copy.

  • Future work includes batching of deltas, data compression and “better support for sparse files.”