Re: [RFC v3] VFIO Migration

[Michael S. Tsirkin]

On Mon, Nov 16, 2020 at 02:38:12PM +0000, Stefan Hajnoczi wrote:
> On Wed, Nov 11, 2020 at 03:41:59PM +0000, Dr. David Alan Gilbert wrote:
> > * Stefan Hajnoczi (stefanha@redhat.com) wrote:
> > > On Wed, Nov 11, 2020 at 12:56:26PM +0000, Dr. David Alan Gilbert wrote:
> > > > * Stefan Hajnoczi (stefanha@redhat.com) wrote:
> > > > > Orchestrating Migrations
> > > > > ------------------------
> > > > > In order to migrate a device a *migration parameter list* must first 
> > > > > be built
> > > > > on the source. Each migration parameter is added to the list if it is 
> > > > > in
> > > > > effect. For example, the migration parameter list for a device with
> > > > > new-feature=off,num-queues=4 would be num-queues=4 if the new-feature 
> > > > > migration
> > > > > parameter was introduced with the off value disabling its effect.
> > > > 
> > > > What component builds that list (i.e. what component needs to know the
> > > > history that new-feature=off was the default - ah I think you answer
> > > > that below).
> > > 
> > > Yep. Thanks for noting this. I'll need to reorder things so it is clear.
> > > 
> > > > > The following conditions must be met to establish migration 
> > > > > compatibility:
> > > > > 
> > > > > 1. The source and destination device model strings match.
> > > > > 
> > > > > 2. Each migration parameter name from the migration parameter list is 
> > > > > supported
> > > > >    by the destination. For example, the destination supports the 
> > > > > num-queues
> > > > >    migration parameter.
> > > > > 
> > > > > 3. Each migration parameter value from the migration parameter list is
> > > > >    supported by the destination. For example, the destination supports
> > > > >    num-queues=4.
> > > > 
> > > > Hmm, are combinations of parameter checks needed - i.e. is it possible
> > > > that a destination supports    num-queues=4 and  new-feature=on/off -
> > > > but only supports new-feature=on when num-queues>2 ?
> > > 
> > > Yes, it's possible but cannot be expressed in the migration info JSON.
> > > 
> > > We need to choose a level of expressiveness that will be useful enough
> > > without being complex. In the extreme the migration info would contain
> > > Turing complete validation expressions (e.g. JavaScript) so that any
> > > relationship can be expressed, but I doubt that complexity is needed.
> > > The other extreme is just booleans and (opaque) strings for maximum
> > > simplicity.
> > > 
> > > If the syntax is not expressive enough then it's impossible to check
> > > migration compatibility without actually creating a new device instance
> > > on the destination. Daniel Berrange raised the requirement of checking
> > > migration compatibility without creating the device since this helps
> > > with selecting a migration destination.
> > 
> > Right, but my worry isn't the JSON description, it's the set of 3
> > conditions above; they need to state that only some combinations need to
> > be valid.
> 
> Yes, the proposed syntax is simply not expressive enough. The migration
> compatibility check will pass and then the destination will refuse to
> set up the device (before the device state is transferred).
> 
> Any suggestions for a syntax without full-blown arithmetic and logic
> expressions?
> 
> > > Any ideas for a better syntax?
> > 
> > I'd be happy with a --param name=value   repeatedly, but also know that
> > some option parsers don't like that.
> 
> Another wart, Sphinx considers repeated options an error so you cannot
> document options using rST option syntax. I remember having this problem
> when documenting virtiofsd's command-line options :).
> 
> If something comes to mind please let me know. I'm not set on a
> particular syntax, but I'd like to choose the one that is both
> human-friendly and compatible with option parsers while avoiding
> namespace collisions with the device implementation's own options.
> 
> Stefan


I think the simplest way is just to include and open-source tool
for figuring all this out together with qemu.
Any vendor interested in supporting migration with qemu
will then just submit a patch for that tool.

This will also help make sure this interface
is not just a way to bypass GPL, we can ask that the
supporting server is opensource.

And it will help us guide vendors towards supporting migration
correctly.

-- 
MST