[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v9 00/10] XBZRLE delta for live migration of lar
From: |
Orit Wasserman |
Subject: |
Re: [Qemu-devel] [PATCH v9 00/10] XBZRLE delta for live migration of large memory app |
Date: |
Thu, 19 Apr 2012 10:16:14 +0300 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120329 Thunderbird/11.0.1 |
On 04/18/2012 08:12 PM, Anthony Liguori wrote:
> Hi,
>
> Just to avoid any 1.1 migration "surprises", I view this series as 1.2
> material. I'm reluctant to make any protocol changes this late in the
> release process (assuming this series got all the necessary review/ack in the
> next 1.5 weeks).
>
> If anyone has a strong feeling differently, now's the time to speak up (as
> opposed to during the -rc phase).
Hi Anthony,
I was indeed aiming this series for 1.2.
Cheers,
Orit
>
> Regards,
>
> Anthony Liguori
>
>
> On 04/11/2012 01:49 PM, Orit Wasserman wrote:
>> Changes from v8:
>> Implement more effiecent cache_resize method
>> fix set_cachesize command comments
>>
>> Changes from v7:
>> Copy current page before encoding it, this will prevents page content
>> change during the encoding.
>> Allow changing the cache size during an active migration.
>> Fix comments by Avi.
>>
>> Changes from v6:
>> 1) add assert checks to ULEB encoding/decoding
>> 2) no need to send last zero run
>>
>> Changes from v5:
>> 1) Add migration capabilities
>> 2) Use ULEB to encode run length
>> 3) Do not send unmodified (dirty) page
>> 3) Fix other patch comments
>>
>> Using GCache or GHashTable requires allocating new buffer on every content
>> change,
>> so I decided to keep the simple cache implementation.
>>
>> Changes from v4:
>> 1) Rebase
>> 2) divide patch into 9 patches
>> 3) move memory allocation into cache_insert
>>
>> Future work :
>> Use SSE for encoding.
>> Page ranking acording to their dirty rate and automatic
>> activation/deactivation of the feature - will be sent in a separate patch
>> series.
>>
>> By using XBZRLE (Xor Based Zero Run Length Encoding) we can reduce VM
>> downtime
>> and total live-migration time of VMs running memory write intensive workloads
>> typical of large enterprise applications such as SAP ERP Systems, and
>> generally
>> speaking for any application with a sparse memory update pattern.
>>
>> The compression format uses the fact that we will have many zero (zero
>> represents
>> an unchanged value).
>> We repesent the page data delta by zero and non zero runs.
>> We represent a zero run with it's length (in bytes).
>> We represent a non zero run with it's length (in bytes) and the data.
>> The run length is encoded using ULEB128 (http://en.wikipedia.org/wiki/LEB128)
>>
>> page = zrun nzrun
>> | zrun nzrun page
>>
>> zrun = length
>>
>> nzrun = length byte...
>>
>> length = uleb128 encoded integer
>>
>> On the sender side XBZRLE is used as a compact delta encoding of page
>> updates,
>> retrieving the old page content from an LRU cache (default size of 512 MB).
>> The
>> receiving side uses the existing page content and XBZRLE to decode the new
>> page
>> content.
>>
>> This is a more compact way to store the delta than the previous version.
>>
>> This work was originally based on research results published VEE 2011:
>> Evaluation of
>> Delta Compression Techniques for Efficient Live Migration of Large Virtual
>> Machines by Benoit, Svard, Tordsson and Elmroth. Additionally the delta
>> encoder
>> XBRLE was improved further using XBZRLE instead.
>>
>> XBZRLE has a sustained bandwidth of 2-2.5 GB/s for typical workloads making
>> it
>> ideal for in-line, real-time encoding such as is needed for live-migration.
>>
>> A typical usage scenario:
>> {qemu} migrate_set_cachesize 256m
>> {qemu} migrate -d -x tcp:destination.host:4444
>> {qemu} info migrate
>> ...
>> transferred ram-duplicate: A kbytes
>> transferred ram-normal: B kbytes
>> transferred ram-xbrle: C kbytes
>> overflow ram-xbrle: D pages
>> cache-miss ram-xbrle: E pages
>>
>> Testing: live migration with XBZRLE completed in 110 seconds, without live
>> migration was not able to complete.
>>
>> A simple synthetic memory r/w load generator:
>> .. include<stdlib.h>
>> .. include<stdio.h>
>> .. int main()
>> .. {
>> .. char *buf = (char *) calloc(4096, 4096);
>> .. while (1) {
>> .. int i;
>> .. for (i = 0; i< 4096 * 4; i++) {
>> .. buf[i * 4096 / 4]++;
>> .. }
>> .. printf(".");
>> .. }
>> .. }
>>
>> Signed-off-by: Benoit Hudzia<address@hidden>
>> Signed-off-by: Petter Svard<address@hidden>
>> Signed-off-by: Aidan Shribman<address@hidden>
>>
>> Orit Wasserman (10):
>> Add cache handling functions
>> Add uleb encoding/decoding functions
>> Add save_block_hdr function
>> Add host_from_stream_offset_versioned function
>> Add MigrationParams structure
>> Add XBZRLE to ram_save_block and ram_save_live
>> Add XBZRLE option to migrate command
>> Add migration capabilites
>> Add set_cachesize command
>> Add XBZRLE statistics
>>
>> arch_init.c | 521
>> ++++++++++++++++++++++++++++++++++++++++++++++++++---
>> block-migration.c | 8 +-
>> hmp-commands.hx | 35 +++-
>> hmp.c | 48 +++++-
>> hmp.h | 2 +
>> migration.c | 92 +++++++++-
>> migration.h | 32 +++-
>> monitor.c | 7 +
>> qapi-schema.json | 64 +++++++-
>> qemu-common.h | 1 +
>> qmp-commands.hx | 78 ++++++++-
>> savevm.c | 133 +++++++++++++-
>> sysemu.h | 3 +-
>> vmstate.h | 2 +-
>> 14 files changed, 969 insertions(+), 57 deletions(-)
>>
>
>
- Re: [Qemu-devel] [PATCH v9 07/10] Add XBZRLE option to migrate command, (continued)