Re: deprecation of in-tree builds

[Daniel P . Berrangé]

On Tue, Mar 31, 2020 at 05:04:16PM +0200, BALATON Zoltan wrote:
> On Tue, 31 Mar 2020, Daniel P. Berrangé wrote:
> > On Tue, Mar 31, 2020 at 02:33:46PM +0200, BALATON Zoltan wrote:
> > > > We will have to ask developers to change habits anyway when we switch to
> > > > Meson.  I agree with Daniel's recommendation to delay changes requiring
> > > > habit-changes until then.  However, telling people to stay clear of the
> > > > unloved and brittle in-tree build is simply good advice we should not
> > > > withhold.
> > > 
> > > Can someone please explain why is it brittle and cannot be supported? It 
> > > has
> > > worked well so far apart from some breakage due to being untested which is
> > > also not a techincal necessity just a decision by some maintiners to not
> > > test it. Adding a CI job to keep it working would also not be difficult or
> > > much complexity.
> > 
> > Writing make rules to correctly handle both src-dir and build-dir scenarios
> > is a non-negligible maint burden. If you look back through QEMU's history
> > we have a steady stream of patches which have broken one or the other
> > build scenarios.
> 
> That's probably becuase there are no clear rules (such as always prefix
> files in source dir with $(SRC_PATH), generated files with some BUILDDIR,
> etc.) and these are not documented so every time someone touches it has to
> explore and debug it again. This could be avoided if these were written down
> once but instead of trying to document and clean up the build system the
> chosen direction is to just throw it out and replace it with something more
> complex and with more dependencies (Meson) and then to save "additional
> complexity" it also breaks people's workflow and demand them to adapt
> themselves. I'm not saying there should be no changes but if there's a way
> to make them less painful it could be considered if it's not much extra work
> and in this case it does not seem to be.

I did indeed write the docs on QEMU's build system found in
docs/devel/build-system.txt  but it is not enough to eliminate
the maint burden of supporting both. QEMU's makefiles are just
too complex and full of unexpected effects to make it easy to
support both srcdir & builddir IMHO.

> > Developers will often not test both scenarios, just the one they prefer
> > to use. This results in a maint burden on the subsystem maintainers who
> > merge patches and then find they break & have to back them out. Sometimes
> > even the subsystem maintainer gets it wrong and burden falls on Peter to
> > find & reject it.
> > 
> > Even if we have CI to test both, it is still a burden on developers to
> > debug failures reported by the CI and figure out what needs fixing. With
> > the number of builds we do & the time for a single CI cycle it gets very
> > time consuming. I've personally wasted many many hours debugging src-dir
> > vs build-dir problems in QEMU's makefiles - probably more than an entire
> > day was lost when I did the patches to split the trace.h header file.
> > 
> > This all serves to divert time away from useful work on QEMU. If there
> > was some critically important functional thing that src-dir builds offer
> > that can't be achieved by build-dir builds, then the extra maint work
> > could be justified. I don't think that's the case though.
> 
> OK, so then only supporting out-of-tree builds but adding convenience
> function and Makefile to still allow people to run configure; make from
> source dir would solve this without also needing people to change what they
> always did so why is that solution not acceptable?

There's several scenarios discussed. Currently we don't do anything to
prevent src-dir builds, so there's no need to extra make magic described
I proposed a patch to add a warning, but it still allows src-dir builds,
so nothing functional has changed.

We could forbid src-dir builds now and add the magic to redirct to a
build-dir. That would work, but I don't think we should block src-dir
builds in make, assuming we have a switch to meson arriving in the
near-ish future.

Once we switch to Meson there will inevitably be some disruption in
build system ussage as it is a completely new tool. That makes it a
good flag day for forbidding src-dir builds. Given that it is a flag
day, I don't think there's any need to create makefile magic at that
time.


IMHO we shouldn't be so afraid of change that we need to try to preserve
every precise way of working we've had in the past. In the long term this
frame of mind holds back the project and promotes a negative impression
of QEMU as an old, outdated project resistant to any kind of change.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|