RE: [PATCH v11 05/11] vfio/migration: Block multiple devices migration

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgg@nvidia.com]
> Sent: 16 May 2023 13:00
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Avihai Horon <avihaih@nvidia.com>; qemu-devel@nongnu.org; Alex
> Williamson <alex.williamson@redhat.com>; Juan Quintela
> <quintela@redhat.com>; Dr. David Alan Gilbert <dgilbert@redhat.com>;
> Michael S. Tsirkin <mst@redhat.com>; Cornelia Huck <cohuck@redhat.com>;
> Paolo Bonzini <pbonzini@redhat.com>; Vladimir Sementsov-Ogievskiy
> <vsementsov@yandex-team.ru>; Cédric Le Goater <clg@redhat.com>; Yishai
> Hadas <yishaih@nvidia.com>; Maor Gottlieb <maorg@nvidia.com>; Kirti
> Wankhede <kwankhede@nvidia.com>; Tarun Gupta <targupta@nvidia.com>;
> Joao Martins <joao.m.martins@oracle.com>
> Subject: Re: [PATCH v11 05/11] vfio/migration: Block multiple devices
> migration
> 
> On Tue, May 16, 2023 at 10:03:54AM +0000, Shameerali Kolothum Thodi
> wrote:
> 
> > > Currently VFIO migration doesn't implement some kind of intermediate
> > > quiescent state in which P2P DMAs are quiesced before stopping or
> > > running the device. This can cause problems in multi-device migration
> > > where the devices are doing P2P DMAs, since the devices are not stopped
> > > together at the same time.
> > >
> > > Until such support is added, block migration of multiple devices.
> >
> > Missed this one. Currently this blocks even if the attached devices are not
> > capable of P2P DMAs. eg; HiSilicon ACC devices. These are integrated end
> point
> > devices without any P2P capability between them. Is it Ok to check for
> > VFIO_MIGRATION_P2P flag and allow if the devices are not supporting
> that?
> 
> Lacking VFIO_MIGRATION_P2P doesn't mean the device is incapable of
> P2P, it means the migration can't support P2P.
> 
> We'd need some kind of new flag to check and such devices should be
> blocked from creating P2P mappings. Basically we don't currently
> fully support devices that are incapable of P2P operations.

Ok. I will take a look.

> What happens on your platform if a guest tries to do P2P? Does the
> platform crash?

I am not sure. Since the devices are behind SMMU, I was under the assumption
that we do have the guarantee of isolation here(grouping). Or this is something
we are worried only during migration?

Thanks,
Shameer