Subject: [PATCH v4] XBZRLE delta for live migration of large memory apps
From: Aidan Shribman<address@hidden>
By using XBZRLE (Xor Binary Zero Run-Length-Encoding) we can reduce VM downtime
and total live-migration time of VMs running memory write intensive workloads
typical of large enterprise applications such as SAP ERP Systems, and generally
speaking for any application with a sparse memory update pattern.
On the sender side XBZRLE is used as a compact delta encoding of page updates,
retrieving the old page content from an LRU cache (default size of 64 MB). The
receiving side uses the existing page content and XBZRLE to decode the new page
content.
Work was originally based on research results published VEE 2011: Evaluation of
Delta Compression Techniques for Efficient Live Migration of Large Virtual
Machines by Benoit, Svard, Tordsson and Elmroth. Additionally the delta encoder
XBRLE was improved further using XBZRLE instead.
XBZRLE has a sustained bandwidth of 2-2.5 GB/s for typical workloads making it
ideal for in-line, real-time encoding such as is needed for live-migration.
A typical usage scenario:
{qemu} migrate_set_cachesize 256m
{qemu} migrate -x -d tcp:destination.host:4444
{qemu} info migrate
...
transferred ram-duplicate: A kbytes
transferred ram-duplicate: B pages
transferred ram-normal: C kbytes
transferred ram-normal: D pages
transferred ram-xbrle: E kbytes
transferred ram-xbrle: F pages
overflow ram-xbrle: G pages
cache-hit ram-xbrle: H pages
cache-lookup ram-xbrle: J pages
Testing: live migration with XBZRLE completed in 110 seconds, without live
migration was not able to complete.
A simple synthetic memory r/w load generator:
.. include<stdlib.h>
.. include<stdio.h>
.. int main()
.. {
.. char *buf = (char *) calloc(4096, 4096);
.. while (1) {
.. int i;
.. for (i = 0; i< 4096 * 4; i++) {
.. buf[i * 4096 / 4]++;
.. }
.. printf(".");
.. }
.. }