suddenly, "recursive make considered harmful" makes piles of sense

[Robert P. J. Day]

  (WARNING: verbosity ahead.)

  a while back when i was just getting into gnu make seriously, i
recall someone pointing me at peter miller's online paper "recursive
make considered harmful".  i skimmed the parts of it i could
understand and thought, "that's cute ... kind of academic ... not sure
how much of it is actually relevant or useful."

  oh, how times have changed.

  over the last couple days, i've developed a real appreciation for
what i *think* he was talking about, and i thought i'd post some
thoughts and get some feedback.  and maybe someone else might find
this useful.

  at the moment (as i've mentioned on previous occasions), i have in
front of me a recursive make structure to build an embedded linux
system, whose structure (simplified) is:

  system/
    kernel/
    rootfs/
      (static stuff)/
      busybox/
      (net stuff)/
        ppp/
        ftp/
        dhcp/
      minit/

and on, and on, and on.  there's lots more, so you get the idea that
it's a fairly big tree with lots of recursion.

  until now, the build structure has involved (not surprisingly) lots
of recursive calls, and makefiles at every level.  which, also not
surprisingly, turns out to be a real pain since if you want to tweak
the build structure, you need to wander down into the directories to
get to the relevant makefile.  a real pain, since every piece of
software that goes into the final system has its own personal,
controlling makefile that's adapted specifically for that software.
and it was just yesterday that it suddenly dawned on me how to make
this much simpler, and only after i started redesigning the structure
did it occur to me that this might be what miller's paper might be
talking about.

  to start, i decided to define a small number of targets that i
wanted to support, say, clean, realclean, configure and build.  that's
a good starting point.  now, *i* want to define everything in terms of
those targets but, of course, different pieces of OSS software may
word their target names differently.  so for some piece of software,
i might have to remap "build" to "compile".  no big deal, it just
means that every makefile has to be custom written.

  after having messed with keeping the makefiles in the directory
structure just above their corresponding software, it suddenly dawned
on me that i could collect them all and put them in a single place --
my OSS makefile repository.  and for each piece of software, i needed
to supply three pieces of info:

  1) location of makefile
  2) location of source directory itself
  3) location of (optional) config file

  imagine i have some software called "foo".  at the very top level
Makefile, i set three variables that will completely define how this
software is to be built:

  export foo_MAKEFILE=
  export foo_SOURCE_DIR=
  export foo_CONFIG_FILE=     (optional if no config file needed)

now (to make a long story ... well, still pretty long), a makefile
that invokes the build of foo would have the target:

  foo:
        ${MAKE} \
                CONFIG_FILE=${foo_CONFIG_FILE} \
                -f ${foo_MAKEFILE} \
                -C ${foo_SOURCE_DIR} \
                ${MAKECMDGOALS}

the only part that's missing is what the makefiles look like for each
piece of software.  well, you do have to write each one individually,
but once they're done, you can archive them in one place and share
them with others -- they're completely reusable.  here's a first draft
of a makefile that would handle busybox:

============================================================
.PHONY: clean realclean configure build

CROSS_COMPILE ?= ppc_8xx-
ARCH ?= ppc

clean:
        ${MAKE} clean           # maps right thru

realclean:
        ${MAKE} distclean       # realclean maps to distclean

configure:
        @if  [ -f ${CURDIR}/.config ] &&                                \
         diff -q ${CONFIG_FILE} ${CURDIR}/.config ; then                \
                echo "No need to configure busybox." ;                  \
        else                                                            \
                cp ${CONFIG_FILE} ${CURDIR}/.config ;                   \
                yes '' | make oldconfig ;                               \
        fi

build: configure
        ${MAKE} \
                CROSS_COMPILER_PREFIX=${CROSS_COMPILE} \
                TARGET_ARCH=${ARCH} \
                busybox
=============================================================

all this does, obviously, is map my choice of targets through to
what targets busybox uses in *its* own makefile.

  this just means that, for every piece of software i want to add to
my overall build, i need to sit down and write its corresponding
makefile, and add it to my makefile collection.  once i do that,
putting together new projects is a piece of cake.  and since all of
the env variables are set at the top-level Makefile, i can tweak any
part of the build in just that one place.  this is making my life
immensely easier already.

  i suspect that, for a lot of you, this is pretty much ho hum, seen
it before, that's pretty basic stuff.  but i figured it might be fun
to share this to see what others have to think.  i doubt this is
*exactly* what miller was talking about, but it's clear that it's
removing a good chunk of the recursive structure from the build
process, and that really does make things easier to work with.

  thoughts?

rday