Re: [Qemu-devel] [PULL 00/34] migration thread and queue

[Paolo Bonzini]

Il 27/12/2012 16:30, Alexandre DERUMIER ha scritto:
> Hi,
> I'm currently testing new migration code with last qemu.git,
> 
> it's working pretty fine (around 30ms downtime with standard workload).
> 
> 
> But I have add some problem, with high memory workload vm. (playing an HD 
> video for example).
> 
> Target vm is pause after migration, 
> # info status
> VM status: paused (internal-error)
> 
> (downtime is around 700ms)

1) What happened before these patches?  If something different, can you
bisect it?

2) Do you get anything on the console (stdout)?  See
kvm_handle_internal_error in kvm-all.c for what to expect.

Paolo

> I can reproduce it 100%
> 
> Regards,
> 
> Alexandre Derumier
> 
> 
> ----- Mail original ----- 
> 
> De: "Juan Quintela" <address@hidden> 
> À: address@hidden 
> Cc: address@hidden 
> Envoyé: Vendredi 21 Décembre 2012 20:41:03 
> Objet: [Qemu-devel] [PULL 00/34] migration thread and queue 
> 
> Hi 
> 
> Changes for last version: 
> - inlined paolo fixes, attached here (they were very small, and only change 
> error cases locking) 
> - I tested with autotest and see no problems 
> 
> Please pull. 
> 
> Thanks, Juan. 
> 
> 
> diff --git a/arch_init.c b/arch_init.c 
> index 86f8544..ea75441 100644 
> --- a/arch_init.c 
> +++ b/arch_init.c 
> @@ -641,13 +641,13 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) 
> } 
> i++; 
> } 
> + qemu_mutex_unlock_ramlist(); 
> 
> if (ret < 0) { 
> bytes_transferred += total_sent; 
> return ret; 
> } 
> 
> - qemu_mutex_unlock_ramlist(); 
> qemu_put_be64(f, RAM_SAVE_FLAG_EOS); 
> total_sent += 8; 
> bytes_transferred += total_sent; 
> @@ -657,9 +657,8 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) 
> 
> static int ram_save_complete(QEMUFile *f, void *opaque) 
> { 
> - migration_bitmap_sync(); 
> - 
> qemu_mutex_lock_ramlist(); 
> + migration_bitmap_sync(); 
> 
> /* try transferring iterative blocks of memory */ 
> 
> diff --git a/migration.c b/migration.c 
> index c69e864..d6ec3e8 100644 
> --- a/migration.c 
> +++ b/migration.c 
> @@ -650,7 +650,7 @@ static int64_t buffered_set_rate_limit(void *opaque, 
> int64_t new_rate) 
> new_rate = SIZE_MAX; 
> } 
> 
> - s->xfer_limit = new_rate / 10; 
> + s->xfer_limit = new_rate / XFER_LIMIT_RATIO; 
> 
> out: 
> return s->xfer_limit; 
> 
> 
> [20121220] 
> Changes for yesterday: 
> - Paolo Acked the series 
> - Rebase on top of today git (only conflicts were due to header re-shuffle) 
> 
> Please pull. 
> 
> [20121219] 
> 
> This is my queue for migration-thread and patches associated. This 
> integrates review comments & code for Paolo. This is the subset from 
> both approachs that we agreed with. rest of patches need more review 
> and are not here. 
> 
> Migrating and idle guest with upstwream: 
> 
> (qemu) info migrate 
> capabilities: xbzrle: off 
> Migration status: completed 
> total time: 34251 milliseconds 
> downtime: 492 milliseconds 
> transferred ram: 762458 kbytes 
> remaining ram: 0 kbytes 
> total ram: 14688768 kbytes 
> duplicate: 3492606 pages 
> normal: 189762 pages 
> normal bytes: 759048 kbytes 
> 
> with this series of patches. 
> 
> (qemu) info migrate 
> capabilities: xbzrle: off 
> Migration status: completed 
> total time: 30712 milliseconds 
> downtime: 29 milliseconds 
> transferred ram: 738857 kbytes 
> remaining ram: 0 kbytes 
> total ram: 14688768 kbytes 
> duplicate: 3503423 pages 
> normal: 176671 pages 
> normal bytes: 706684 kbytes 
> 
> Notice the big difference in downtime. And that is also seen inside 
> the guest a program that just do an idle loop seeing how "long" it 
> takes to wait for 10ms. 
> 
> with upstream: 
> 
> address@hidden ~]# ./timer 
> delay of 452 ms 
> delay of 114 ms 
> delay of 136 ms 
> delay of 135 ms 
> delay of 136 ms 
> delay of 131 ms 
> delay of 134 ms 
> 
> with this series of patches, wait never takes 100ms, nothing is printed. 
> 
> Please pull. 
> 
> Thanks, Juan. 
> 
> The following changes since commit 27dd7730582be85c7d4f680f5f71146629809c86: 
> 
> Merge remote-tracking branch 'bonzini/header-dirs' into staging (2012-12-19 
> 17:15:39 -0600) 
> 
> are available in the git repository at: 
> 
> 
> git://repo.or.cz/qemu/quintela.git thread.next 
> 
> for you to fetch changes up to 381c08083929f50f4780ea272ea36f7e5899b3b6: 
> 
> migration: merge QEMUFileBuffered into MigrationState (2012-12-21 20:01:24 
> +0100) 
> 
> ---------------------------------------------------------------- 
> Juan Quintela (25): 
> migration: include qemu-file.h 
> migration-fd: remove duplicate include 
> buffered_file: Move from using a timer to use a thread 
> migration: make qemu_fopen_ops_buffered() return void 
> migration: move migration thread init code to migrate_fd_put_ready 
> migration: make writes blocking 
> migration: remove unfreeze logic 
> migration: just lock migrate_fd_put_ready 
> buffered_file: Unfold the trick to restart generating migration data 
> buffered_file: don't flush on put buffer 
> buffered_file: unfold buffered_append in buffered_put_buffer 
> savevm: New save live migration method: pending 
> migration: move buffered_file.c code into migration.c 
> migration: add XFER_LIMIT_RATIO 
> migration: move migration_fd_put_ready() 
> migration: Inline qemu_fopen_ops_buffered into migrate_fd_connect 
> migration: move migration notifier 
> ram: rename last_block to last_seen_block 
> ram: Add last_sent_block 
> memory: introduce memory_region_test_and_clear_dirty 
> ram: Use memory_region_test_and_clear_dirty 
> ram: optimize migration bitmap walking 
> ram: account the amount of transferred ram better 
> ram: refactor ram_save_block() return value 
> migration: merge QEMUFileBuffered into MigrationState 
> 
> Paolo Bonzini (7): 
> migration: fix migration_bitmap leak 
> buffered_file: do not send more than s->bytes_xfer bytes per tick 
> migration: remove double call to migrate_fd_close 
> exec: change ramlist from MRU order to a 1-item cache 
> exec: change RAM list to a TAILQ 
> exec: sort the memory from biggest to smallest 
> migration: fix qemu_get_fd for BufferedFile 
> 
> Umesh Deshpande (2): 
> add a version number to ram_list 
> protect the ramlist with a separate mutex 
> 
> Makefile.objs | 3 +- 
> arch_init.c | 244 +++++++++++++------------- 
> block-migration.c | 49 ++---- 
> buffered_file.c | 268 ----------------------------- 
> buffered_file.h | 22 --- 
> dump.c | 8 +- 
> exec.c | 128 +++++++++----- 
> include/exec/cpu-all.h | 15 +- 
> include/exec/memory.h | 16 ++ 
> include/migration/migration.h | 13 +- 
> include/migration/qemu-file.h | 5 - 
> include/migration/vmstate.h | 1 + 
> include/sysemu/sysemu.h | 1 + 
> memory.c | 16 ++ 
> memory_mapping.c | 4 +- 
> migration-exec.c | 3 +- 
> migration-fd.c | 4 +- 
> migration-tcp.c | 3 +- 
> migration-unix.c | 3 +- 
> migration.c | 390 +++++++++++++++++++++++++++++++----------- 
> savevm.c | 24 ++- 
> target-i386/arch_dump.c | 2 +- 
> 22 files changed, 599 insertions(+), 623 deletions(-) 
> delete mode 100644 buffered_file.c 
> delete mode 100644 buffered_file.h 
> 
>