Parallel jobs with order-only prerequisites

[Steven Simpson]

Hi,

I'm using GNU Make 3.81 to compile a Java source tree, and extract what 
is essentially a header file of its public methods and fields, i.e. the 
parts of the source tree that another tree might depend on.  I'll refer 
to this header file as a 'profile', and the set of classes in a source 
tree as a 'module'.

In general, for a module 'foo', making the target foo.done first causes 
the module to be compiled, then the profile to be extracted. The profile 
is saved in foo.api, but only if the content changes - so, even though 
foo.done is re-stamped on each compilation, foo.api retains its earlier 
timestamp if its content doesn't change.

If a module 'foo' depends on the profile of 'bar', it is expressed with 
two rules:

foo.done: | bar.done
foo.done: bar.api

There is no rule with bar.api as a target, but the /preceding/ 
order-only rule will ensure that bar.done is up-to-date, and therefore 
that bar.api exists, before the second rule is tried, i.e. before 
bar.api's timestamp is read.

Here's a simulation of 'compilation' for three modules, with foo 
depending on bar and baz:

all:: foo.done

%.done: %.src
        @echo Starting "$*"
        @sleep 3
        @tr -d ' \n' < "$<" > "$*.api-tmp"
        @cmp -s "$*.api" "$*.api-tmp" || \
                ( cp "$*.api-tmp" "$*.api" ; echo "$*" API changed )
        @touch "$@"
        @echo Completed "$*"

foo.done: | bar.done baz.done
foo.done: bar.api baz.api

clean::
        $(RM) *~
        $(RM) *.done
        $(RM) *.api
        $(RM) *.api-tmp

You need to create {foo,bar,baz}.src, which simulate source trees. The 
simulated profile-extraction process simply strips out spaces, so that 
it's possible for one source change to lead to a profile change while 
another does not.

All this works as long as targets are met serially, but fails with 'make 
-j', as it attempts to meet some part of {bar,baz}.{done,api} in 
parallel.  On a clean build, bar and baz start compiling at once, but 
the test for bar.api is made before the completion of bar's compilation 
and profile extraction, so building of foo.done fails:

$ make -j
Starting bar
make: *** No rule to make target `baz.api', needed by `foo.done'. Stop.
Starting baz
make: *** Waiting for unfinished jobs....
bar API changed
baz API changed
Completed bar
Completed baz

A subsequent 'make -j' completes the process.

At other stages, the timestamps of {bar,baz}.api have already been 
compared with foo.done before they get updated, so if bar.src is 
modified in a profile-changing way, bar is recompiled, but foo isn't, 
until Make is invoked a second time.

Can anything be done to work around this?  I tried adding rules such as:

bar.api: | bar.done
baz.api: | baz.done

...but this doesn't seem to prevent the (foo.done older-than bar.api) 
test from happening too soon when bar.api already exists. You avoid the 
error from a clean build, but still require the double invocation of 
Make later on.

Is there a way, in both serial and parallel contexts, to force bar.done 
to complete before the timestamp of bar.api is tested, with the version 
of Make I'm using?  Is there an alternative strategy?

Cheers,

Steven