Re: Parallel jobs with order-only prerequisites

[Steven Simpson]

Dawid,

On 13/10/14 21:56, Dawid Gosławski wrote:
> I was talking about something like that:
>
> all:: | foo.done
>     @echo "\nCompilation finished !!!\n"
>     @echo "----\nListing timestamps:"
>     @find ./ -name "*api*" -printf "%f - %c\n" | sort
>
> DATA := "BASIC API"
> FILES :=foo.src bar.src baz.src
>
> %.src:
>     @touch $@
>
> %.done: %.src
>     @echo "1-$(@F)) Starting"
>     @echo "  1.1-$(@F)) Making"
>     @echo $(DATA) >  $(patsubst %.done,%.api-tmp, $@) # Of course it's 
> only for example
>     @echo "  1.2-$(@F)) Generated api-tmp"
>     @echo "  1.3-$(@F)) COMPILED !!!"

Does this not lack a 'touch $@' to mark the "compilation" complete? 
Otherwise all modules get built every time.

(If so, should it go before or after creating %.api-tmp, as the two are 
compared in the next rule?)

> # Ensure that api exist only AFTER compilation of %.done succeded
> # Because
> # Empty cause "done" step created api-tmp (for comparing with old one) 
> we use api it should also exists at this stage - provided by done
> %.api-tmp: %.done
>     @echo "2.0-$(@F)) Api-tmp hit so $^ is ready and $(patsubst 
> %.api-tmp,%.api, $@) too !!!"
>     @cmp -s "$@" "$(patsubst %.api-tmp,%.api, $@)" || \
>         ( cp "$@" "$(patsubst %.api-tmp,%.api, $@)" ; echo "  3.1) $@" 
> API COPIED!!! )
>
> foo.done: bar.done baz.done bar.api-tmp baz.api-tmp

Don't the first two prerequisites mean that foo is compiled every time 
bar or baz is?  My aim is to prevent that, only compiling foo if the 
/profiles/ of bar or baz change.

Anyway, I think I get the point about separating the cmp-cp from 
compilation, and it has lead me to this:

all:: foo.done

%.done: %.src
        @echo Starting "$*"
        @sleep 2
        @touch "$@"
        @tr -d ' \n' < "$<" > "$*.api-tmp"
        @echo Completed "$*"

bar.api: | bar.done
baz.api: | baz.done
foo.api: | foo.done

%.api: %.done
        @echo Testing API of "$*"
        @cmp -s "$*.api" "$*.api-tmp" || \
                ( cp "$*.api-tmp" "$*.api" ; echo "$*" API changed )


foo.done: bar.api baz.api

clean::
        $(RM) *~
        $(RM) *.done
        $(RM) *.api
        $(RM) *.api-tmp

This seems to work in both serial and parallel.

There is a small disadvantage, in that the cmp-cp is done on every 
module that failed to produce a profile change last time, but these 
operations aren't very time-consuming.  It also means that you don't get 
a re-assuring "Nothing to be done for `all'", even though nothing was 
practically done.  (Is there a way to stop the building of %.api being 
counted as doing something?)

I'll see if I can incorporate it into my framework...

Thanks,

Steven