RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang, Chen]

> -----Original Message-----
> From: Daniel P. Berrangé <berrange@redhat.com>
> Sent: Monday, June 20, 2022 6:37 PM
> To: Zhang, Chen <chen.zhang@intel.com>
> Cc: Thomas Huth <thuth@redhat.com>; Jason Wang
> <jasowang@redhat.com>; qemu-dev <qemu-devel@nongnu.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Eduardo Habkost <eduardo@habkost.net>;
> Eric Blake <eblake@redhat.com>; Markus Armbruster
> <armbru@redhat.com>; Peter Maydell <peter.maydell@linaro.org>; Laurent
> Vivier <lvivier@redhat.com>; Yuri Benditovich
> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> submodule for QEMU
> 
> On Mon, Jun 20, 2022 at 10:29:14AM +0000, Zhang, Chen wrote:
> >
> >
> > > -----Original Message-----
> > > From: Thomas Huth <thuth@redhat.com>
> > > Sent: Monday, June 20, 2022 5:44 PM
> > > To: Zhang, Chen <chen.zhang@intel.com>; Daniel P. Berrangé
> > > <berrange@redhat.com>
> > > Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > > devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > > Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > > Markus Armbruster <armbru@redhat.com>; Peter Maydell
> > > <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>;
> > > Yuri Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > > <andrew@daynix.com>
> > > Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
> > > as a submodule for QEMU
> > >
> > > On 20/06/2022 11.29, Zhang, Chen wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Thomas Huth <thuth@redhat.com>
> > > >> Sent: Monday, June 20, 2022 4:47 PM
> > > >> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
> > > >> <chen.zhang@intel.com>
> > > >> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > > >> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > > >> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > > Markus
> > > >> Armbruster <armbru@redhat.com>; Peter Maydell
> > > >> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>;
> > > >> Yuri Benditovich <yuri.benditovich@daynix.com>; Andrew
> > > >> Melnychenko <andrew@daynix.com>
> > > >> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf
> > > >> project as a submodule for QEMU
> > > >>
> > > >> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> > > >>> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> > > >>>>
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: Daniel P. Berrangé <berrange@redhat.com>
> > > >>>>> Sent: Friday, June 17, 2022 4:05 PM
> > > >>>>> To: Zhang, Chen <chen.zhang@intel.com>
> > > >>>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > > >>>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>;
> > > >>>>> Eduardo Habkost <eduardo@habkost.net>; Eric Blake
> > > >>>>> <eblake@redhat.com>; Markus Armbruster
> <armbru@redhat.com>;
> > > >>>>> Peter Maydell <peter.maydell@linaro.org>; Thomas Huth
> > > >>>>> <thuth@redhat.com>;
> > > >> Laurent
> > > >>>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
> > > >>>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > > >>>>> <andrew@daynix.com>
> > > >>>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf
> > > >>>>> project as a submodule for QEMU
> > > >>>>>
> > > >>>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > > >>>>>> Make iovisor/ubpf project be a git submodule for QEMU.
> > > >>>>>> It will auto clone ubpf project when configure QEMU.
> > > >>>>>
> > > >>>>> I don't think we need todo this. As it is brand new
> > > >>>>> functionality we don't have any back compat issues. We should
> > > >>>>> just expect the distros to ship ubpf if they want their QEMU
> > > >>>>> builds to take advantage
> > > of it.
> > > >>>>>
> > > >>>>
> > > >>>> Yes, agree. It's the best way to use the uBPF project.
> > > >>>> But current status is distros(ubuntu, RHEL...) does not ship
> > > >>>> the iovisor/ubpf like the iovisor/bcc. So I have to do it.
> > > >>>> Or do you have any better suggestions?
> > > >>>
> > > >>> If distros want to support the functionality, they can add
> > > >>> packages for it IMHO.
> > > >>
> > > >> Yes, let's please avoid new submodules. Submodules can sometimes
> > > >> be a real PITA (e.g. if you forget to update before rsync'ing
> > > >> your code to a machine that has limited internet access), and if
> > > >> users install QEMU from sources, they can also install ubpf from
> sources, too.
> > > >> And if distros want to support this feature, they can package
> > > >> ubpf on their own, as Daniel said.
> > > >
> > > > Hi Daniel and Thomas,
> > > >
> > > > I don't know much the background history of QEMU submodules, but
> > > meson
> > > > build is a submodule for QEMU too. It means user can't install
> > > > QEMU from sources with limited internet access.
> > >
> > > There is no written policy, but I think the general consensus is
> > > that we only ship code in submodules if:
> > >
> > > 1) It's not available in a required version in distros yet
> > >
> > > and
> > >
> > > 2) it is essentially required to build QEMU (like meson) or if the
> > > feature has been part of the QEMU sources before and then moved to a
> > > separate repository (like slirp).
> > >
> > > We ship meson as a submodule since we require some meson features
> > > that are not available with the meson versions in the distros yet.
> > > Once the distros catch up, we'll likely remove the meson submodule from
> QEMU.
> > >
> > > > And back to Daniel's comments,  Yes, the best way is distros add
> > > > the ubpf packages, But maybe it's too late to implement new
> > > > features for us. We can introduce the submodule now and auto
> > > > change to the distros's
> > > lib when distros add it.  For example QEMU's submodule SLIRP do it
> > > in the same way.
> > >
> > > slirp used to be part of the QEMU repository, but then has been
> > > moved to a separate project a while ago. However, at that point in
> > > time there weren't any packages ins distros yet, so we had to include it 
> > > as
> a submodule.
> > >
> > > Now that the distros ship it, too, we're planning to remove the
> > > slirp submodule from QEMU soon, see:
> > >
> > >
> > > https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html
> > >
> > > > It make user experience the latest technology with no other
> > > > dependencies.
> > >
> > > Well, that's only true if we update the submodule in QEMU regularly.
> > > If we forget to update, we could easily miss some important (maybe
> > > even security
> > > related) fixes from the upstream projects. This can be a nightmare
> > > for distros, when they then have to go around and look into each and
> > > every projects whether they embed a certain code module that needs a
> > > CVE fix. It's better if it can be fixed in one central spot instead.
> > >
> > > > uBPF infrastructure have the ability to extend the capabilities
> > > > without requiring changing source code. If we not allow it, we
> > > > have to re-implement all the eBPF assembler, disassembler,
> > > > interpreter, and JIT
> > > compiler like DPDK userspace eBPF support (DPDK can't use ubpf
> > > project by license issue).
> > >
> > > Not sure whether I understood that statement right ... nobody said
> > > that QEMU should not allow it - we just suggested to rely on a
> > > system installation of ubpf instead of embedding the code. Or is
> > > that not possible?? (I don't know that project yet - isn't it
> > > possible to compile it as a shared library?)
> >
> > Thanks for your details explanation.
> > It looks better to introduce the uBPF shared library for QEMU.
> > For example:
> > ./configure --ubpf-lib=path
> 
> I've not looked, so maybe it already does this, but ideally  'uBPF'
> would ship a 'pkg-config'  file, so that apps can automatically find it and 
> set
> the right cflags/libs etc for the compiler. For configure integration, 
> normally
> we'd expect it to be --enable-ubpf/--disable-ubpf, with it automatically
> enabling itself if the pkg-config file is found.
> Take a look at handling of some existing libraries we depend on for examples.
> 

OK, thanks your comments. I reviewed uBPF code, but no pkg-config file.
The make install just copy the libubpf.so/.a to /usr/lib.
Fortunately, writing this file looks easy. I will try to do it and make QEMU 
use the libubpf.so like "pixman".

Thanks
Chen
 

> With regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|