Re: [PATCH Kernel v24 0/8] Add UAPIs to support migration for VFIO devices

[Xiang Zheng]

Hi Alex,

Thank you for your suggestion.

On 2020/7/22 6:43, Alex Williamson wrote:
> On Tue, 21 Jul 2020 10:43:21 +0800
> Xiang Zheng <zhengxiang9@huawei.com> wrote:
> 
>> Hi Kirti,
>>
>> Sorry to disturb you since this patch set has been merged, and I cannot
>> receive the qemu-side emails about this patch set.
>>
>> We are going to support migration for VFIO devices which support dirty
>> pages tracking.
>>
>> And we also plan to leverage SMMU HTTU feature to do the dirty pages
>> tracking for the devices which don't support dirty pages tracking.
>>
>> For the above two cases, which side determines to choose IOMMU driver or
>> vendor driver to do dirty bitmap tracking, Qemu or VFIO?
>>
>> In brief, if both IOMMU and VFIO devices support dirty pages tracking,
>> we can check the capability and prefer to track dirty pages on device
>> vendor driver which is more efficient.
>>
>> The qusetion is which side to do the check and selection? In my opinion,
>> Qemu/userspace seems more suitable.
> 
> Dirty page tracking is consolidated at the vfio container level.
> Userspace has no basis for determining or interface for selecting a
> dirty bitmap provider, so I would disagree that QEMU should play any
> role here.  The container dirty bitmap tries to provide the finest
> granularity available based on the support of all the devices/groups
> managed by the container.  If there are groups attached to the
> container that have not participated in page pinning, then we consider
> all DMA mappings within the container as persistently dirty.  Once all
> of the participants subscribe to page pinning, the dirty scope is
> reduced to the pinned pages.  IOMMU support for dirty page logging would
> introduce finer granularity yet, which we would probably prefer over
> page pinning, but interfaces for this have not been devised.

Kevin and his colleagues may add these APIs in the future.
We also plan to support these interfaces on SMMU driver and afterwards we
can have a further discussion.

> 
> Ideally userspace should be unaware of any of this, the benefit would
> be seen transparently by having a more sparsely filled dirty bitmap,
> which more accurately reflects how memory is actually being dirtied.

Yes, indeed.

-- 
Thanks,
Xiang