[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 2/7] multifd: modifying 'migrate' qmp command to add multi
From: |
Markus Armbruster |
Subject: |
Re: [PATCH v2 2/7] multifd: modifying 'migrate' qmp command to add multifd socket on particular src and dest pair |
Date: |
Tue, 02 Aug 2022 09:53:57 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Het Gala <het.gala@nutanix.com> writes:
> On 26/07/22 4:43 pm, Daniel P. Berrangé wrote:
>> On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote:
>>> i) Modified the format of the qemu monitor command : 'migrate' by adding a
>>> list,
>>> each element in the list consisting of multifd connection parameters:
>>> source
>>> uri, destination uri and of the number of multifd channels between each
>>> pair.
>>>
>>> ii) Information of all multifd connection parameters' list and length of the
>>> list is stored in 'OutgoingMigrateParams' struct.
>>>
>>> Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
>>> Signed-off-by: Het Gala <het.gala@nutanix.com>
>>> ---
>>> migration/migration.c | 52 +++++++++++++++++++++++++++++--------
>>> migration/socket.c | 60 ++++++++++++++++++++++++++++++++++++++++---
>>> migration/socket.h | 19 +++++++++++++-
>>> monitor/hmp-cmds.c | 1 +
>>> qapi/migration.json | 47 +++++++++++++++++++++++++++++----
>>> 5 files changed, 160 insertions(+), 19 deletions(-)
>>>
>>> diff --git a/qapi/migration.json b/qapi/migration.json
>>> index 81185d4311..456247af8f 100644
>>> --- a/qapi/migration.json
>>> +++ b/qapi/migration.json
>>> @@ -1449,12 +1449,37 @@
>>> ##
>>> { 'command': 'migrate-continue', 'data': {'state': 'MigrationStatus'} }
>>> +##
>>> +# @MigrateUriParameter:
>>> +#
>>> +# Information regarding which source interface is connected to which
>>> +# destination interface and number of multifd channels over each interface.
>>> +#
>>> +# @source-uri: uri of the source VM. Default port number is 0.
>>> +#
>>> +# @destination-uri: uri of the destination VM
>>> +#
>>> +# @multifd-channels: number of parallel multifd channels used to migrate
>>> data
>>> +# for specific source-uri and destination-uri. Default
>>> value
>>> +# in this case is 2 (Since 7.1)
>>> +#
>>> +##
>>> +{ 'struct' : 'MigrateUriParameter',
>>> + 'data' : { 'source-uri' : 'str',
>>> + 'destination-uri' : 'str',
>>> + '*multifd-channels' : 'uint8'} }
>>> +
>>> ##
>>> # @migrate:
>>> #
>>> # Migrates the current running guest to another Virtual Machine.
>>> #
>>> # @uri: the Uniform Resource Identifier of the destination VM
>>> +# for migration thread
>>> +#
>>> +# @multi-fd-uri-list: list of pair of source and destination VM Uniform
>>> +# Resource Identifiers with number of multifd-channels
>>> +# for each pair
>>> #
>>> # @blk: do block migration (full disk copy)
>>> #
>>> @@ -1474,20 +1499,32 @@
>>> # 1. The 'query-migrate' command should be used to check migration's
>>> progress
>>> # and final result (this information is provided by the 'status'
>>> member)
>>> #
>>> -# 2. All boolean arguments default to false
>>> +# 2. The uri argument should have the Uniform Resource Identifier of
>>> default
>>> +# destination VM. This connection will be bound to default network
>>> #
>>> -# 3. The user Monitor's "detach" argument is invalid in QMP and should not
>>> +# 3. All boolean arguments default to false
>>> +#
>>> +# 4. The user Monitor's "detach" argument is invalid in QMP and should not
>>> # be used
>>> #
>>> # Example:
>>> #
>>> -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } }
>>> +# -> { "execute": "migrate",
>>> +# "arguments": {
>>> +# "uri": "tcp:0:4446",
>>> +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900",
>>> +# "destination-uri": "tcp:0:4480",
>>> +# "multifd-channels": 4},
>>> +# { "source-uri": "tcp:10.0.0.0: ",
>>> +# "destination-uri": "tcp:11.0.0.0:7789",
>>> +# "multifd-channels": 5} ] } }
>>> # <- { "return": {} }
>>> #
>>> ##
>>> { 'command': 'migrate',
>>> - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool',
>>> - '*detach': 'bool', '*resume': 'bool' } }
>>> + 'data': {'uri': 'str', '*multi-fd-uri-list': ['MigrateUriParameter'],
>>> + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool',
>>> + '*resume': 'bool' } }
>>
>> Considering the existing migrate API from a QAPI design POV, I
>> think there are several significant flaws with it
>>
>> The use of URIs is the big red flag. It is basically a data encoding
>> scheme within a data encoding scheme. QEMU code should be able to
>> directly work with the results from QAPI, without having todo a
>> second level of parsing.
Concur.
>> URIs made sense in the context of HMP or the QemuOpts CLI, but do not
>> make sense in QMP. We made a mistake in this respect when we first
>> introduced QMP and implemented 'migrate'.
>>
>> If we going to extend the migrate API I think we should stop using URIs
>> for the new fields, and instead define a QAPI discriminated union for
>> the different data transport backends we offer.
>>
>> { 'enum': 'MigrateTransport',
>> 'data': ['socket', 'exec'] }
>>
>> { 'union': 'MigrateAddress',
>> 'base': { 'transport': 'MigrateTransport'},
>> 'discriminator': 'transport',
>> 'data': {
>> 'socket': 'SocketAddress',
>> 'exec': ['str'],
>> }
>>
>> NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock'
>> and 'fd' already. I'm fuzzy on best way to represent RDMA.
>>
>>
>> IIUC, the desire of migration maintainers is that we can ultimately
>> have multifd as the preferred, or even only, mechanism. Aside from
>> the main outbound migration control channel, and the multifd
>> data channels, IIUC we have a potential desire to have more channels
>> for post-copy async requests.
>>
>> This all suggests to me a more general representation along the
>> lines of:
>>
>> { 'enum': 'MigrateChannelType',
>> 'data': ['control', 'data', 'async'] }
>>
>> { 'struct': 'MigrateChannel',
>> 'data': {
>> 'type': 'MigrateChannelType',
>> 'src-addr': 'MigrateAddress',
>> 'dst-addr': 'MigrateAddress',
>> 'count': 'int',
>> } }
>>
>> { 'comand': 'migrate',
>> 'data': {
>> '*uri': 'str'
>> '*channels': ['MigrateChannel']
>> }
>> }
>>
>> With 'uri' and 'channels' being mutually exclusive here.
>>
>> This whole scheme brings in redundancy wrt to the 'migrate-set-parameters'
>> API wrt multifd - essentally the same data is now being set in two
>> different places. IMHO, we should declare the 'multifd' capability
>> and the 'multifd-chanels' parameter deprecated, since the information
>> they provide is totally redundant, if you're giving an explicit list
>> of channels to 'migrate'.
>
> Hi Daniel. Initially while brainstorming this idea for the first time, we
> also came up with the same thought of depricating the migrate
> API. But how will we achieve this now and how is it going to work. Is it like
> we will be making migate V2 APIs initially, integrate it and then
> depricate the old one? would be happy to get some pointers from your end.
Migration maintainers, please advise.
- Re: [PATCH v2 2/7] multifd: modifying 'migrate' qmp command to add multifd socket on particular src and dest pair,
Markus Armbruster <=