Re: [RFC PATCH v1 5/7] vhost-user-blk: add mechanism to track the guest notifiers init state

[Dima Stepanov]

On Wed, Apr 22, 2020 at 12:17:50PM -0400, Raphael Norwitz wrote:
> There are some problems with this patch. It doesn't apply cleanly.
> 
> Are you sure you’re developing on an up to date master branch?
> 
> On Thu, Apr 23, 2020 at 09:39:36PM +0300, Dima Stepanov wrote:
> > 
> > In case of the vhost-user devices the daemon can be killed at any
> > moment. Since QEMU supports the reconnet functionality the guest
> > notifiers should be reset and disabled after "disconnect" event. The
> > most issues were found if the "disconnect" event happened during vhost
> > device initialization step.
> > The disconnect event leads to the call of the vhost_dev_cleanup()
> > routine. Which memset to 0 a vhost device structure. Because of this, if
> > device was not started (dev.started == false) and the connection is
> > broken, then the set_guest_notifier method will produce assertion error.
> > Also connection can be broken after the dev.started field is set to
> > true.
> > A new notifiers_set field is added to the vhost_dev structure to track
> > the state of the guest notifiers during the initialization process.
> > 
> > Signed-off-by: Dima Stepanov <address@hidden>
> > ---
> >  hw/block/vhost-user-blk.c |  8 ++++----
> >  hw/virtio/vhost.c         | 11 +++++++++++
> >  include/hw/virtio/vhost.h |  1 +
> >  3 files changed, 16 insertions(+), 4 deletions(-)
> > 
> >  @@ -1417,6 +1422,8 @@ void vhost_dev_disable_notifiers(struct vhost_dev 
> > *hdev, VirtIODevice *vdev)
> 
> I can’t find the function vhost_dev_drop_guest_notifiers() in
> /hw/virtio/vhost.c, or anywhere in the codebase.
> 
> Where does this code come from?

These wrapper functions were introduced in this patch set with the
previous patch:
  [RFC PATCH v1 4/7] vhost: introduce wrappers to set guest notifiers for
virtio device
The problem was found in the vhost-user-blk stack. But it seems to me
that it is the same for all vhost-user devices. That is why i've tried
to suggest solution which will resolve other vhost-user devices, too.
And this was the reason to add new wrappers.

> 
> >          virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index 
> > + i);
> >      }
> >      virtio_device_release_ioeventfd(vdev);
> > +
> > +    hdev->notifiers_set = false;
> >  }
> >  
> >  /*
> > @@ -1449,6 +1456,10 @@ int vhost_dev_drop_guest_notifiers(struct vhost_dev 
> > *hdev,
> >      VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
> >      int ret;
> >  
> > +    if (!hdev->notifiers_set) {
> > +        return 0;
> > +    }
> > +
> >      ret = k->set_guest_notifiers(qbus->parent, nvqs, false);
> >      if (ret < 0) {
> >          error_report("Error reset guest notifier: %d", -ret);
> > diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> > index 4d0d2e2..e3711a7 100644
> > --- a/include/hw/virtio/vhost.h
> > +++ b/include/hw/virtio/vhost.h
> > @@ -90,6 +90,7 @@ struct vhost_dev {
> >      QLIST_HEAD(, vhost_iommu) iommu_list;
> >      IOMMUNotifier n;
> >      const VhostDevConfigOps *config_ops;
> > +    bool notifiers_set;
> >  };
> >  
> >  int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
> > -- 
> > 2.7.4
> > 
> >