more general question on parallelization and checkpoints

[Robert P. J. Day]

  i've been messing with this all afternoon, and i can't help but
think there's an easier way to do this than what i've come up with.
here's the general problem, which is an extension of what i asked
about earlier.

  i have a top-level makefile which has to support multiple
subdirectories and multiple actions -- for example:

  SUBDIRS = d1 d2 d3 d4
  ACTIONS = clean configure build install

  now, i know there are a couple different ways to write the recursive
processing.  the first doesn't support parallel processing (all of the
following is drastically simplified so, yes, it's kind of
pseudo-code):

  ${ACTIONS}:
        for d in ${SUBDIRS} ; do        \
                ${MAKE} -C $$d $@ ;     \
        done

  the second technique is what i've been using until now -- to support
*potential* future parallelization:

  .PHONY: ${SUBDIRS}
  ${SUBDIRS}:
        ${MAKE} -C $@ ${MAKECMDGOALS}


  now, here's the problem.  i'd like to be able to define, for any of
the actions, a possible pre-processing and possible post-processing
target.  as an example, say all of my subdirectories have an "install"
target that installs stuff into a single destination directory.

  however, before *anyone* can do an install, i might want to remove
the existing dest. directory and initialize a new one.  so i would
have an "install" pre-processing target that would have to be run
before *any* subdirectory install target is processed.

  similarly, perhaps i want to define an install *post*-processing
step -- maybe run through the entire output directory, strip
executables, fix symlinks and so on.  (yes, that could possibly be
done by *each* install sub-target, but there are some cases where it's
just *way* easier to wait until all children have done their install,
then take care of it at the end.)

  i'm trying to do this with the second (parallelizable) approach
above and it's just getting ugly.  at this point, i'm ready to throw
in the towel and go back to the non-parallelizable loop approach,
which would certainly be easier to modify.

  am i asking too much here?  is there a standard way to do this sort
of thing?

rday

  p.s.  to summarize, in a perfect world, each sub-directory would be
able to be processed in parallel with every other sub-directory but,
if i define a pre-processing step for *any* action, then no
sub-directory would be allowed to start processing that action until
the pre-processing step was complete.  and the pre-processing step
would *only* be run if at least one subdirectory was processing that
action.  (i.e., there would be no point running the "install"
pre-processing step if no subdirectory was processing the "install"
action.)

  similarly, if i defined a post-processing step for an action, then
that post-processing would not be allowed to run until all
subdirectories has processed that action.

  i originally thought this wasn't going to be that difficult, but i'm
rapidly changing my mind.

  thoughts?