Re: [PATCH v2 1/1] docs/devel: Add VFIO device migration documentation

[Dr. David Alan Gilbert]

* Daniel P. Berrangé (berrange@redhat.com) wrote:
> On Thu, Mar 11, 2021 at 12:50:09AM +0530, Tarun Gupta wrote:
> > Document interfaces used for VFIO device migration. Added flow of state 
> > changes
> > during live migration with VFIO device. Tested by building docs with the new
> > vfio-migration.rst file.
> > 
> > v2:
> > - Included the new vfio-migration.rst file in index.rst
> > - Updated dirty page tracking section, also added details about
> >   'pre-copy-dirty-page-tracking' opt-out option.
> > - Incorporated comments around wording of doc.
> > 
> > Signed-off-by: Tarun Gupta <targupta@nvidia.com>
> > Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> > ---
> >  MAINTAINERS                   |   1 +
> >  docs/devel/index.rst          |   1 +
> >  docs/devel/vfio-migration.rst | 135 ++++++++++++++++++++++++++++++++++
> >  3 files changed, 137 insertions(+)
> >  create mode 100644 docs/devel/vfio-migration.rst
> 
> 
> > +Postcopy
> > +========
> > +
> > +Postcopy migration is not supported for VFIO devices.
> 
> What is the problem here and is there any plan for how to address it ?

There's no equivalent to userfaultfd for accesses to RAM made by a
device.
There's some potential for this to be doable with an IOMMU or the like,
but:
  a) IOMMUs and devices aren't currently happy at recovering from
failures
  b) the fragementation you get during a postcopy probably isn't pretty
when you get to build IOMMU tables.

> Postcopy is essentially the only migration mechanism that can reliably
> complete, so it really should be considered the default approach to
> migration for all mgmt apps wanting to do migration, except in special
> cases.   IOW, if we want VFIO migration to be viable, we need postcopy
> support.

There's lots of other things postcopy doesn't work with; so hmm.

Dave

> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK