Re: [Qemu-devel] Tunneled Migration with Non-Shared Storage

[Paolo Bonzini]

On 19/11/2014 10:35, Dr. David Alan Gilbert wrote:
> * Paolo Bonzini (address@hidden) wrote:
>>
>>
>> On 18/11/2014 21:28, Dr. David Alan Gilbert wrote:
>>> This seems odd, since as far as I know the tunneling code is quite separate
>>> to the migration code; I thought the only thing that the migration
>>> code sees different is the file descriptors it gets past.
>>> (Having said that, again I don't know storage stuff, so if this
>>> is a storage special there may be something there...)
>>
>> Tunnelled migration uses the old block-migration.c code.  Non-tunnelled
>> migration uses the NBD server and block/mirror.c. 
> 
> OK, that explains that.  Is that because the tunneling code can't 
> deal with tunneling the NBD server connection?
> 
>> The main problem with
>> the old code is that uses a possibly unbounded amount of memory in
>> mig_save_device_dirty and can have huge jitter if any serious workload
>> is running in the guest.
> 
> So that's sending dirty blocks iteratively? Not that I can see
> when the allocations get freed; but is the amount allocated there
> related to total disk size (as Gary suggested) or to the amount
> of dirty blocks?

It should be related to the maximum rate limit (which can be set to
arbitrarily high values, however).

The reads are started, then the ones that are ready are sent and the
blocks are freed in flush_blks.  The jitter happens when the guest reads
a lot but only writes a few blocks.  In that case, the bdrv_drain_all in
mig_save_device_dirty can be called relatively often and it can be
expensive because it also waits for all guest-initiated reads to complete.

The bulk phase is similar, just with different functions (the reads are
done in mig_save_device_bulk).  With a high rate limit, the total
allocated memory can reach a few gigabytes indeed.

Depending on the scenario, a possible disadvantage of NBD migration is
that it can only throttle each disk separately, while the old code will
apply a single limit to all migrations.

Paolou