Re: Time to introduce a migration protocol negotiation (Re: [PATCH v2 00/25] migration: Postcopy Preemption)

[Peter Xu]

On Wed, Mar 16, 2022 at 11:00:53AM +0000, Daniel P. Berrangé wrote:
> On Wed, Mar 16, 2022 at 06:40:08PM +0800, Peter Xu wrote:
> > On Wed, Mar 16, 2022 at 09:59:28AM +0000, Daniel P. Berrangé wrote:
> > > On Wed, Mar 16, 2022 at 11:30:59AM +0800, Peter Xu wrote:
> > > > On Tue, Mar 15, 2022 at 11:15:41AM +0000, Daniel P. Berrangé wrote:
> > > > > > I still remember you mentioned the upper layer softwares can have
> > > > > > assumption on using only 1 pair of socket for migration, I think 
> > > > > > that makes
> > > > > > postcopy-preempt by default impossible.
> > > > > > 
> > > > > > Why multifd is different here?
> > > > > 
> > > > > It isn't different. We went through the pain to extending libvirt
> > > > > to know how to open many channels for multifd. We'll have todo
> > > > > the same with this postcopy-pre-empt. To this day though, management
> > > > > apps above libvirt largely don't enable multifd, which is a real
> > > > > shame. This is the key reason I think we need to handle this at
> > > > > the QEMU level automatically.
> > > > 
> > > > But I still don't undertand how QEMU could know about those tunnels, 
> > > > which
> > > > should be beyond QEMU's awareness?
> > > > 
> > > > The tunneling program can be some admin initiated socat tcp forwarding
> > > > programs, which by default may not allow >1 socket pairs.
> > > > 
> > > > Or maybe I have mis-understood on what's the tunneling we're discussing?
> > > 
> > > I dont think I was talking about tunneling at all, just QEMU
> > > migration protocol options !
> > 
> > Ah. :)
> > 
> > > 
> > > If an app is tunnelling QEMU's migration protocol over some
> > > channel, that isn't important to QEMU - regardless whether a
> > > passed in 'fd:' protocol FD is a direct TCP socket, or a
> > > UNIX socket for a tunnel, QEMU works the same way. In one
> > > of my other replies I mention a way to make 'fd:' work with
> > > an arbitrary number of channels, by using an event from QEMU
> > > to request the app provide additional FDs.
> > 
> > I very much agree on the whole concept of what you proposed, either on the
> > new negotiation phase itself, or the idea that with the negotiation phase
> > we can try to auto-enable some features we not used to.
> > 
> > What I wanted to express is we can't enable either preempt mode or multifd
> > automatically from qemu even with them, because these two are quite special
> > IMHO in that qemu doesn't know whether the mgmt app can handle the multiple
> > socket pairs.  Yes we could teach qemu to dynamically accept new "fd"s, but
> > again IMHO that still needs to be intervened by the mgmt app.
> 
> My proposal absolutely *can* let QEMU do that automatically, and that
> is one of the most important benefits of it.
> 
> [quote]
> Introduce one *final-no-more-never-again-after-this* migration
> capability called "protocol-negotiation".
> 
> When that capability is set, first declare that henceforth the
> migration transport is REQUIRED to support **multiple**,
> **bi-directional** channels. We might only use 1 TCP channel
> in some cases, but it declares our intent that we expect to be
> able to use as many channels as we see fit henceforth.
> [/quote]
> 
> IOW, any management app that enabled 'protocol-negotiation' is explicitly
> declaring that it accepts the new requirements for support for multiple
> channels. An app which enabled 'protocol-negotiation' capability while
> only allowing 1 chanels is simply broken, because it would be violating
> the documented requirements for the capability.

Sorry I misteriously overlooked that paragraph.. it's just that from the
wording "negotiation" shouldn't rely on multipe sockets, since from the
literal meaning any bidirectional channel should be negotiatable.  But I
see what you mean now..  Thanks,

-- 
Peter Xu