Re: [Automake-NG] [PATCH 00/11] Several changes to parallel-tests support

[Akim Demaille]

Le 9 mai 2012 à 13:11, Stefano Lattarini a écrit :

>> I have one worry though: what about runtime performances?  Sure,
>> late binding provides more features, but it might be costly.  Do
>> you have means to evaluate that we have no serious regressions?
>> 
> Not really (even if I didn't notice any slowdown in during use of
> the Automake testsuite); do you want me to run some measurements?
> 
> (And this makes me think: should we add a "performance" testsuite
> in Automake, offering a quick and convenient way to measure how a
> changeset impacts performances?  And if yes: would you have any
> suggestion on how to set this up?)

Yes, I think you should at least try to make some measurements.
Deeply nested Make macros can be extremely costly, especially
because they are not lazy, this is really call by name: when used
several times, they are evaluated several times.  There must be
some good reason for this, but then result is longer compilation,
which is _bad_.

I can't find the references ATM, but I remember I read about
some build system at Google that removes all this dynamic aspect
that exist in Makefiles, which resulted in a much lighter impact
of the tool itself on the build time.

> But back to the point: notice that the pre-existing implementation of
> the parallel testsuite harness already made quite an extensive use of
> late bindings in the definition of $(TESTS) (to allow things like
> overriding of TESTS and TEST_LOGS at make runtime), and this has never
> had any negative impact on the performances.

I was not referring especially to this, but for instance the fact
that uniqueness of files to distribute is now on the responsability
of $(sort) instead of being a static cost.  Again, that's just an
example, and probably one where this added cost is probably negligible,
as it should appear only for dist* targets.

Consider this as a general concern.  Plain "make all" should be as fast
as possible, so beware of macros that could be repeatedly used in
a regular build.  Tests and dist are definitely less critical (well,
especially dist).


>> Also, I don't know if it's part of your specs for automake-ng,
>> but, performance-wise *and* from the point of view of semantics,
>> *and* of syntax, I am very fond of this new feature of GNU Make 3.82:
>> 
>> * New special target: .ONESHELL instructs make to invoke a single instance
>>  of the shell and provide it with the entire recipe, regardless of how many
>>  lines it contains.
>> 
> Ugh, no please.  Some times it is very useful to split a recipe in distinct
> shell invocations, and the '.ONESHELL' directive would prevent us from
> doing so (as, AFAIK, it acts on a global level, not on a target-specific
> level).
> 
> More importantly: I want Automake-NG to work with at least GNU make 3.81 as
> well (and also GNU make 3.80, it that's not too difficult), so using a
> feature introduced in 3.82 is a big no-no.

While I completely agree with, and expected, your second argument,
I confess I do not understand the first one.  IMHO it should have been
the rule from day 1 in Make that the whole recipe be evaluated by
a single shell invocation, with set -e.  The obfuscation of the
recipes is painful.  So I rather think that it should be the opposite:
the much rarer cases where "it is very useful to split a recipe in distinct
shell invocations" should be the ones that are (slightly) harder to write.

I agree that going to this model is a completely change, but hey,
if Automake-ng does not make this move, I believe it will just
never happen.

I'm not trying to force anything here, I'm just grieving for something
I really like.