Re: [Qemu-devel] Should memory hotplug work with vhost-user backends?

[Dr. David Alan Gilbert]

* Raphael Norwitz (address@hidden) wrote:
> > > I've briefly looked through the libvhost-user code and the hot-add path
> > > looks safe enough to me (or at least no more broken than the regular
> > > vhost-user memory hot-add path).
> > >
> > > Can you elaborate a little more about why memory hot-add is unsafe with
> > > vhost-user device backends built with libvhost-user, as opposed to those
> > > using the raw vhost-user protocol semantics?
> >
> > The libvhost-user problem is that the library is mostly designed for a
> > single-threaded event loop that handles all virtqueue and vhost-user
> > protocol activity.
> >
> > As soon as virtqueues are handled by dedicated threads there are race
> > conditions between the virtqueue threads and the vhost-user protocol
> > thread.
> >
> > A virtqueue thread may or may not have an up-to-date view of memory
> > translation.  This can result in it missing memory that is currently
> > being hotplugged and continuing to access memory that has been removed.
> >
> 
> I agree - this is definitely seems like a problem if memory is being removed,
> but I don’t see how a virtqueue thread may have an outdated view of memory
> in the hot-add case.

In the vhost-user client it gets a bit tricky because the setmemtable
command doesn't give differences - it gives a whole mapping table; so
the way libvhost-user does it is to perform a whole new set of mappings.
While the qemu and kernel side doesn't see any change, the mappings do
change in the client.  This probably needs a protocol improvement.

> libvhost-user now supports the REPLY_ACK feature, so that on hot-add qemu
> will wait for a response from the backend, confirming the new memory was
> successfully mapped in, before returning from vhost_user_set_mem_table(). If
> the new memory is mapped in by the backend before the ram is exposed to the
> guest, how could a virtqueue thread receive operations on missing memory?

Yeh, I think use of that new memory isn't that bad.

Dave

> > Dave might have additional comments.
> >
> > Stefan
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK