Re: [PATCH/RFC 0/1] Vhost User Cross Cable: Intro

[Nikos Dragazis]

On Tue, 14 Jan 2020 at 10:20 Stefan Hajnoczi
<address@hidden> wrote:
> On Fri, Jan 10, 2020 at 10:34 AM Marc-André Lureau
> <address@hidden> wrote:
> > On Wed, Jan 8, 2020 at 5:57 AM V. <address@hidden> wrote:
>
> Hi V.,
> I think I remember you from Etherboot/gPXE days :).
>
> > > 3.
> > > Now if Cross Cable is actually a new and (after a code-rewrite of 10) a
> > > viable way to connect 2 QEMU's together, could I actually
> > > suggest a better way?
> > > I am thinking of a '-netdev vhost-user-slave' option to connect (as client
> > > or server) to a master QEMU running '-netdev vhost-user'.
> > > This way there is no need for any external program at all, the master can
> > > have it's memory unshared and everything would just work
> > > and be fast.
> > > Also the whole thing can fall back to normal virtio if memory is not 
shared
> > > and it would even work in pure usermode without any
> > > context switch.
> > >
> > > Building a patch for this idea I could maybe get around to, don't clearly
> > > have an idea how much work this would be but I've done
> > > crazier things.
> > > But is this is something that someone might be able to whip up in an hour
> > > or two? Someone who actually does have a clue about vhost
> > > and virtio maybe? ;-)
> >
> > I believe https://wiki.qemu.org/Features/VirtioVhostUser is what you
> > are after. It's still being discussed and non-trivial, but not very
> > active lately afaik.
>
> virtio-vhost-user is being experimented with in the SPDK community but
> there has been no activity on VIRTIO standardization or the QEMU
> patches for some time.  More info here:
> https://ndragazis.github.io/spdk.html
>
> I think the new ivshmem v2 feature may provide the functionality you
> are looking for, but I haven't looked at them yet.  Here is the link:
> https://www.mail-archive.com/address@hidden/msg668749.html
>
> And here is Jan's KVM Forum presentation on ivshmem v2:
> https://www.youtube.com/watch?v=TiZrngLUFMA
>
> Stefan


Hi Stefan,

First of all, sorry for the delayed response. The mail got lost
somewhere in my inbox. Please keep Cc-ing me on all threads related to
virtio-vhost-user.

As you correctly pointed out, I have been experimenting with
virtio-vhost-user on SPDK and [1] is a working demo on this matter. I
have been working on getting it merged with SPDK and, to this end, I
have been interacting with the SPDK team [2][3] and mostly with Darek
Stojaczyk (Cc-ing him).

The reason that you haven’t seen any activity for the last months is
that I got a job and hence my schedule has become tighter. But I will
definitely find some space and fit it into my schedule. Let me give you
a heads up, so that we get synced:

Originally, I created and sent a patch (influenced from your DPDK patch
[4]) to SPDK that was enhancing SPDK’s internal rte_vhost library with
the virtio-vhost-user transport. However, a few weeks later, the SPDK
team decided to switch from their internal rte_vhost library to using
DPDK’s rte_vhost library directly [3]. Given that, I refactored and sent
the patch for the virtio-vhost-user transport to the DPDK mailing list
[5]. Regarding the virtio-vhost-user device, I have made some
enhancements [6] on your original RFC device implementation and, as you
may remember, I have sent some revised versions of the spec to the
virtio mailing list [7].

At the moment, the blocker is the virtio spec. The last update on this
was my discussion with Michael Tsirkin (Cc-ing him as well) about the
need for the VIRTIO_PCI_CAP_DOORBELL_CFG and
VIRTIO_PCI_CAP_NOTIFICATION_CFG configuration structures [8].

So, I think the next steps should be the following:

1. merging the spec
2. adding the device on QEMU
3. adding the vvu transport on DPDK
4. extending SPDK to make use of the new vhost-user transport

To conclude, I still believe that this device is useful and should be
part of virtio/qemu/dpdk/spdk and I will continue working on this
direction.

Best regards,
Nikos


[1] https://ndragazis.github.io/spdk.html
[2] 
https://lists.01.org/hyperkitty/list/address@hidden/thread/UR4FM45LEQIBJNQ4MTDZFH6SLTXHTGDR/#ZGPRKS47QWHXHFBEKSCA7Z66E2AGSLHN
[3] 
https://lists.01.org/hyperkitty/list/address@hidden/thread/WLUREJGPK5UJVTHIQ5GRL3CDWR5NN5BI/#G7P3D4KF6OQDI2RYASXQOZCMITKT5DEP
[4] http://mails.dpdk.org/archives/dev/2018-January/088155.html
[5] https://lore.kernel.org/dpdk-dev/address@hidden/T/
[6] https://lists.gnu.org/archive/html/qemu-devel/2019-04/msg02910.html
[7] https://lists.oasis-open.org/archives/virtio-dev/201906/msg00036.html
[8] https://lists.oasis-open.org/archives/virtio-dev/201908/msg00014.html