Re: [Gnu-arch-users] Build System links/ recommendations

[Jan Hudec]

On Fri, Sep 03, 2004 at 13:11:15 +0100, Andrew Suffield wrote:
> On Fri, Sep 03, 2004 at 11:12:00AM +0200, Jan Hudec wrote:
> > On the other hand, the inclusive make should never (provided it's
> > written reasonably) do worse, that recursive one.
> 
> There are plenty of pathological cases where monolithic makefiles
> perform worse. They force make to traverse an immense DAG for every
> single operation.
> 
> There is no reason to think that these cases don't occur naturally.

Um. You are right. Now I see it. If the projects consists of several
independent or mostly independent parts, than separate makefile for each
of them should perform better, because it only stats appropriate
dependencies each time.

> > > > We wrote a school project. It was not big, but it had a part in
> > > > userland, part in kernel, part for windows and parts that were compiled
> > > > for more than one part (the windows part had it's own build system for
> > > > MSVC++ though).
> > > > 
> > > > First we had standard recursive makefiles (hand written GNU makefiles). 
> > > > It was slow, it kept relinking it all the time (because the recursive
> > > > make was considered a change, though it wasn't) and we had problems with
> > > > things not getting rebuilt several times.
> > > > 
> > > > Then I rewrote the whole system for inclusive make (hand written GNU
> > > > makefiles again). It was faster by an order of magnitude and reliable.
> > > 
> > > This sort of story is the reason why that paper has become so
> > > popular. There's a problem with it:
> > > 
> > > You replaced a broken build system with a less broken build system,
> > > which coincidentally happened to use a monolithic makefile.
> > > 
> > > You don't really have any way to know *which* changes mattered. As a
> > > general rule, whenever you rewrite a broken build system, it will get
> > > faster and better.
> > 
> > Yes, we DO know, at least partialy. We identified two problems, that
> > could only be solved by inclusive make:
> > 
> > 1) There is no way to tell make, that though it ran a rule, the rule
> >    didn't change anything. Thus the objects that depend on things built
> >    by recursive make are always rebuilt.
> 
> Here's the way that doesn't exist:
> 
> foo.o: foo.c
> 
> foo.so: | foo.o
> 
> The behaviour you describe is what you normally want. In the strange
> scenarios where you don't want cascading rebuilds, you use an
> order-only prerequisite, which behaves like "make foo.o && make foo.so".
> 
> I can't imagine how you think monolithic makefiles are different
> here. The cascade behaviour is the same either way. The way to turn it
> off is also the same.

Um, we failed to find that in the GNU Make texinfo documentation. But it
may be.

The cascading behaviour is NOT the same. In the recursive makefile, the
binary was always linked, even if nothing at all was touched (because
the first level invocation didn't know dependencies of the library and
had to call the second make).

> > 2) The headers have to be checked by each invocation of make. Since
> >    we had many headers, included by each other quite extensively, the
> >    dependency-checking was significant.
> 
> That's strange, and doesn't make any sense. This is what disk caches
> are for. I'd have investigated in more detail; the scenario you
> describe is not supposed to occur. stat()ting the same file twice in a
> row should not take noticably longer than stat()ting it once.
> 
> I've seen plenty of projects with large numbers of cross-including
> headers and I've never seen a problem like you describe here. So I
> suspect there was something strange going on with your makefiles.

It's not really in a row. And well, it was the overhead of calling make
as a whole. The makefiles were all quite large and they all used
wildcards and funny stuff like that. The inclusive makefile was shorter
because it only stated the complicated stuff once.

Anyway: Would you agree with a sumary, that:
    * For mostly independent parts of a project, ie. source vs. doc vs.
      l18n stuff etc. recursive makefiles perform better (and are easier
      to maintain). The same usualy holds for sources of libraries.
    * For things that depend on each other a lot, single makefile does
      better job.
?

-------------------------------------------------------------------------------
                                                 Jan 'Bulb' Hudec 
<address@hidden>

signature.asc
Description: Digital signature