Re: Time to introduce a migration protocol negotiation (Re: [PATCH v2 00/25] migration: Postcopy Preemption)

[Daniel P . Berrangé]

On Wed, Mar 16, 2022 at 06:40:08PM +0800, Peter Xu wrote:
> On Wed, Mar 16, 2022 at 09:59:28AM +0000, Daniel P. Berrangé wrote:
> > On Wed, Mar 16, 2022 at 11:30:59AM +0800, Peter Xu wrote:
> > > On Tue, Mar 15, 2022 at 11:15:41AM +0000, Daniel P. Berrangé wrote:
> > > > > I still remember you mentioned the upper layer softwares can have
> > > > > assumption on using only 1 pair of socket for migration, I think that 
> > > > > makes
> > > > > postcopy-preempt by default impossible.
> > > > > 
> > > > > Why multifd is different here?
> > > > 
> > > > It isn't different. We went through the pain to extending libvirt
> > > > to know how to open many channels for multifd. We'll have todo
> > > > the same with this postcopy-pre-empt. To this day though, management
> > > > apps above libvirt largely don't enable multifd, which is a real
> > > > shame. This is the key reason I think we need to handle this at
> > > > the QEMU level automatically.
> > > 
> > > But I still don't undertand how QEMU could know about those tunnels, which
> > > should be beyond QEMU's awareness?
> > > 
> > > The tunneling program can be some admin initiated socat tcp forwarding
> > > programs, which by default may not allow >1 socket pairs.
> > > 
> > > Or maybe I have mis-understood on what's the tunneling we're discussing?
> > 
> > I dont think I was talking about tunneling at all, just QEMU
> > migration protocol options !
> 
> Ah. :)
> 
> > 
> > If an app is tunnelling QEMU's migration protocol over some
> > channel, that isn't important to QEMU - regardless whether a
> > passed in 'fd:' protocol FD is a direct TCP socket, or a
> > UNIX socket for a tunnel, QEMU works the same way. In one
> > of my other replies I mention a way to make 'fd:' work with
> > an arbitrary number of channels, by using an event from QEMU
> > to request the app provide additional FDs.
> 
> I very much agree on the whole concept of what you proposed, either on the
> new negotiation phase itself, or the idea that with the negotiation phase
> we can try to auto-enable some features we not used to.
> 
> What I wanted to express is we can't enable either preempt mode or multifd
> automatically from qemu even with them, because these two are quite special
> IMHO in that qemu doesn't know whether the mgmt app can handle the multiple
> socket pairs.  Yes we could teach qemu to dynamically accept new "fd"s, but
> again IMHO that still needs to be intervened by the mgmt app.

My proposal absolutely *can* let QEMU do that automatically, and that
is one of the most important benefits of it.

[quote]
Introduce one *final-no-more-never-again-after-this* migration
capability called "protocol-negotiation".

When that capability is set, first declare that henceforth the
migration transport is REQUIRED to support **multiple**,
**bi-directional** channels. We might only use 1 TCP channel
in some cases, but it declares our intent that we expect to be
able to use as many channels as we see fit henceforth.
[/quote]

IOW, any management app that enabled 'protocol-negotiation' is explicitly
declaring that it accepts the new requirements for support for multiple
channels. An app which enabled 'protocol-negotiation' capability while
only allowing 1 chanels is simply broken, because it would be violating
the documented requirements for the capability.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|