Re: [PATCH RFC] migration: warn about non-migratable configurations unless '--no-migration' was specified

[Eduardo Habkost]

On Tue, Apr 20, 2021 at 03:10:44PM +0100, Dr. David Alan Gilbert wrote:
> * Eduardo Habkost (ehabkost@redhat.com) wrote:
> > On Tue, Apr 20, 2021 at 12:51:43PM +0100, Dr. David Alan Gilbert wrote:
> > > * Eduardo Habkost (ehabkost@redhat.com) wrote:
> > > > On Mon, Apr 19, 2021 at 07:47:34PM +0100, Dr. David Alan Gilbert wrote:
> > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote:
> > > > > > On Mon, Apr 19, 2021 at 06:15:56PM +0100, Daniel P. Berrangé wrote:
> > > > > > > On Mon, Apr 19, 2021 at 06:11:47PM +0100, Dr. David Alan Gilbert 
> > > > > > > wrote:
> > > > > > > > * Eduardo Habkost (ehabkost@redhat.com) wrote:
> > > > > > > > > I would make live migration policy an enum, just to make sure
> > > > > > > > > we are explicit about the requirements:
> > > > > > > > > 
> > > > > > > > > - UNKNOWN: this is the current state in QEMU 6.0, where we 
> > > > > > > > > don't
> > > > > > > > >   really know what the user expects.
> > > > > > > > >   This can be the default on existing versioned machine types,
> > > > > > > > >   just for compatibility.
> > > > > > > > >   I suggest making this print warnings for every migration
> > > > > > > > >   blocker (like this patch does).
> > > > > > > > >   I suggest deprecating this behavior as soon as we can.
> > > > > > > > > 
> > > > > > > > > - PREFERRED: try to make the VM migratable when possible, but
> > > > > > > > >   don't print a warning or error out if migration is blocked.
> > > > > > > > >   This seems to be the behavior expected by libvirt today.
> > > > > > > > > 
> > > > > > > > > - NOT_NEEDED: live migration is not needed, and QEMU is free 
> > > > > > > > > to
> > > > > > > > >   enable features that block live migration or change guest 
> > > > > > > > > ABI.
> > > > > > > > >   We can probably make this the default on machine types that
> > > > > > > > >   never supported live migration.
> > > > > > > > 
> > > > > > > > I suggest you could do this by adding:
> > > > > > > >   -warn-none-migratable
> > > > > > > >   -no-warn-none-migratable
> > > > > > > > 
> > > > > > > > and then argue about defaults another time.
> > > > > > > 
> > > > > > > If we're going to add new args, lets at least future proof our
> > > > > > > approach with an extensible option that we can wire into QMP
> > > > > > > too later
> > > > > > > 
> > > > > > >   -migratable  none|preferred|required 
> > > > > > > 
> > > > > > > and letting us add extra key/value pairs to tune it if desired.
> > > > > > 
> > > > > > Having said that, we potentially don't need a dedicated arg if we
> > > > > > just make  'migratable=none|preferred|required' be a property of
> > > > > > the machine type and hook everything off that
> > > > > 
> > > > > I think my only difficulty with that is that I don't find any of those
> > > > > 3 words 'obvious'.
> > > > 
> > > > Any suggestions of replacements for those 3 words?
> > > > 
> > > > Would the descriptions below be enough to clarify their meaning
> > > > in documentation?
> > > 
> > > I prefer things that are fairly obvious without needing to look at the
> > > documentation until you want the detail.
> > > 
> > > > - NONE: live migration is not needed, and device or machine code
> > > >   is allowed to enable features that block live migration or
> > > >   change guest ABI.
> > > >   (Not implemented yet)
> > > > 
> > > > - PREFERRED: machine and device code should try to make the VM
> > > >   migratable when possible, but won't emit a warning or error out
> > > >   if migration is blocked.
> > > >   (Current default behavior)
> > > > 
> > > > - REQUIRED: live migration support is required, and adding a
> > > >   migration blocker will be an error.
> > > >   (Implemented today by --only-migratable)
> > > 
> > > How about
> > >   -migratable blocked
> > >      Live migration is not allowed; an outbound migration will fail
> > 
> > "none" and NOT_NEEDED above were about letting QEMU automatically
> > enable features that block live migration or change guest ABI.
> > 
> > If that's implied by "blocked", I'd like to get that documented
> > explicitly.  If that's not implied by "blocked", I don't
> > understand what's the use case for "blocked".
> 
> My 'blocked' is stronger - migration is hard disabled by a blocker
> always; it's for (rare) cases where the user wants to stop a migration
> happening, even if qemu believes it can do it.

The main point of my "not_needed" suggestion was to get
convenient defaults for use cases where live migration and guest
ABI stability is not needed.

I don't know yet if that's an implicit feature of your definition
of "blocked", or not.

> 
> > > 
> > >   -migratable allowed
> > >      Live migration is allowed, but some devices/options may block
> > >      it if they're unable to migrate [current default]
> > 
> > "preferred" above was about QEMU trying to keep live migration
> > working as much as possible.  That's something we all expect QEMU
> > to do, but it's not documented anywhere.
> > 
> > If that's implied by "allowed", I'd like to get that documented
> > explicitly.  If that's not implied by "allowed", then we have a
> > problem.
> 
> My difficulty by your definition is I don't understand what
> 'working as much as possible' means - that's the current behaviour
> as I understand it.   I think mine is more explicit.

There's something I don't see documented in your proposal, which
I've tried to describe it as "try to keep live migration working
as much as possible".  I'll try a more detailed description:

  - Live migration will be possible unless some specific devices
    or features are enabled.
  - Devices or features that block live migration won't be
    enabled by default.
  - Devices or features that break guest ABI won't be enabled by
    default.
  - If an existing configuration allows live migration on a given
    QEMU version, the same configuration will allow live
    migration in newer QEMU versions.
  - If an existing configuration allows live migration with a
    given machine type version, the same configuration will allow
    live migration in a newer version of that machine type.

If all of the above is implied by "allowed", I'd like to get this
documented explicitly.  If it's not, we still have a problem.

-- 
Eduardo