Re: [Qemu-devel] [PATCH 0/4] Add ignore-external migration capability

[Yury Kotov]

11.01.2019, 23:09, "Dr. David Alan Gilbert" <address@hidden>:
> * Yury Kotov (address@hidden) wrote:
>>  10.01.2019, 23:12, "Dr. David Alan Gilbert" <address@hidden>:
>>  > * Yury Kotov (address@hidden) wrote:
>>  >> Hi,
>>  >>
>>  >> The series adds migration capability which allows to skip 'external' RAM 
>> blocks
>>  >> during migration. External block is a RAMBlock which available from the 
>> outside
>>  >> of current QEMU process (e.g. file in /dev/shm). It's useful for fast 
>> local
>>  >> migration to update QEMU for the running guests.
>>  >
>>  > Hi Yury,
>>  > There have been a few similar patch series around from people wanting
>>  > to do similar things.
>>  > In particular Lai Jiangshan's 
>> https://lists.nongnu.org/archive/html/qemu-devel/2018-03/msg07511.html
>>  > and Cédric Le Goater wanted to skip regions for a different reason.
>>  >
>>  > We merged some of Cédric's code last year so that we now
>>  > have the qemu_ram_is_migratable() function - and we should be reusing
>>  > that to skip things rather than adding a new check that we have to add
>>  > everywhere.
>>  >
>>
>>  I didn't see the series, so I'll check it, thanks!
>>  But I saw qemu_ram_is_migratable() function and corresponding patch.
>>  It's very close to my needs, but it works a bit different IIUC:
>>  1. Not migratable blocks isn't validated (existence and size) during 
>> migration,
>>  2. "Migratable" state is determined during the block creation time.
>>     Such case isn't valid because of it:
>>     * Source has one migratable and one not migratable RAM blocks,
>>     * Target has the same (idstr) blocks, but both are not migratable.
>>     Thus, target will not expect pages for not migratable blocks.
>
> I've added Cédric to the mail;
> there were other cases people were interested in, including switching
> it dynamically, just no one else used it yet.
>
> I'd prefer it if you did modify the is_migratable - that will fix a lot
> of other places in the migration code to avoid your blocks; I think the
> best thing is for you to add a spearate 'needs_migration_check' for
> those blocks like yours which you want to check but you don't send the
> data. Please don't change the format of the migratino stream except
> in the case where you are sending those to be checked.
>

Ok, I've got your point and will try to reuse/modify is_migratable.

>>  > Also, ypu're skipping 'external' things, I think the other suggestion
>>  > was to skip 'shared' things (i.e. anything with share=0); skipping
>>  > share=on cases sounds easier to me.
>>
>>  I agree that introducing new term is a complication, but 'share' and 
>> 'external'
>>  terms have important differences (I'll describe it below).
>>
>>  Just to clarify:
>>  * 'share' means that other processes has an access to such memory,
>>  * 'external' means file backed memory.
>>
>>  There is another use case I wanted to support (I had to write about it in
>>  the cover letter, sorry..):
>>  1. Migrate source VM to file and kill source,
>>  2. Start target VM and migrate it from file.
>>  In such case source VM may have memory-backend-ram with share=off, it's ok.
>>
>>  Thus, in the new migration capability I want to migrate memory that meets
>>  three conditions:
>>  1. The source will not use the memory after migration ends,
>>  2. The source may exit before target starts (migrate to file),
>>  3. The target has an access to the memory.
>>
>>  I think 'external' fits them better than 'share'.
>
> Are you sure that with share=off (the default), in the case where the
> source shuts down, that the changes have been written to the backing
> file?
>

Oh, you're right. I was sure it would work...

> I'm also wondering if perhaps we'd be better having an explicit
> migrate=off property on memory objects rather than trying to guess from
> the share= or the fact it's an external path.

It's good idea. Perhaps this is the best option, given the above.

>
> Igor: Does that make sense to you?
>
> Dave
>
>>  >
>>  > Dave
>>  >
>>  >> Patches:
>>  >> 1. Add offset validation to make sure that external RAM block has the 
>> same
>>  >> physical offset on target side,
>>  >> 2. Add RAM_EXTERNAL flag to determine external RAM blocks,
>>  >> 3. Add ignore-external migration capability,
>>  >> 4. Add a test.
>>  >>
>>  >> Usage example:
>>  >> 1. Start source VM:
>>  >> qemu-system-x86 \
>>  >> -m 4G \
>>  >> -object 
>> memory-backend-file,id=mem0,size=4G,share=on,mem-path=/dev/shm/mem0 \
>>  >> -numa node,memdev=mem0 \
>>  >> -qmp unix:/tmp/qemu-qmp-1.sock,server,nowait \
>>  >>
>>  >> 2. Start target VM:
>>  >> qemu-system-x86 \
>>  >> -m 4G \
>>  >> -object 
>> memory-backend-file,id=mem0,size=4G,share=on,mem-path=/dev/shm/mem0 \
>>  >> -numa node,memdev=mem0 \
>>  >> -qmp unix:/tmp/qemu-qmp-2.sock,server,nowait \
>>  >> -incoming defer
>>  >>
>>  >> 3. Enable ignore-external capability on both VMs:
>>  >> { "execute": "migrate-set-capabilities" , "arguments":
>>  >> { "capabilities": [ { "capability": "x-ignore-external", "state": true } 
>> ] } }
>>  >>
>>  >> 4. Start migration.
>>  >>
>>  >> Regards,
>>  >> Yury
>>  >>
>>  >> Yury Kotov (4):
>>  >> migration: add RAMBlock's offset validation
>>  >> exec: add RAM_EXTERNAL flag to mark non-QEMU allocated blocks
>>  >> migration: introduce ignore-external capability
>>  >> tests/migration-test: Add a test for ignore-external capability
>>  >>
>>  >> backends/hostmem-file.c | 3 +-
>>  >> exec.c | 7 ++-
>>  >> include/exec/cpu-common.h | 1 +
>>  >> include/exec/memory.h | 3 ++
>>  >> migration/migration.c | 9 ++++
>>  >> migration/migration.h | 1 +
>>  >> migration/ram.c | 52 ++++++++++++++++--
>>  >> numa.c | 4 +-
>>  >> qapi/migration.json | 6 ++-
>>  >> tests/migration-test.c | 109 +++++++++++++++++++++++++++++++-------
>>  >> 10 files changed, 165 insertions(+), 30 deletions(-)
>>  >>
>>  >> --
>>  >> 2.20.1
>>  > --
>>  > Dr. David Alan Gilbert / address@hidden / Manchester, UK
>>
>>  Regards,
>>  Yury
> --
> Dr. David Alan Gilbert / address@hidden / Manchester, UK

Regards,
Yury