Re: Paralizing configure

[Ralf Wildenhues]

* Paul Eggert wrote on Tue, Feb 08, 2011 at 09:39:21PM CET:
> On 02/08/11 02:55, Marian Marinov wrote:
> 
> > My idea is to split the tests that configure dose into different sections. 
> > And 
> > paralize the tests in each seaction. Then if one test returns false the 
> > whole 
> > section should be considered for retesting.
> 
> Although that might help a bit, I expect it's too much work, because people
> will have to go through by hand and parallelize their code.  I expect it'd be
> better if 'configure' were parallelized by default, much as makefiles are,
> with the programmers mentioning explicit dependencies as necessary.

It's harder than it looks.  Maybe AC_CHECK_PROG checks are independent
from one another.  And we intended the AC_CHECK*_ONCE checks to be so,
but I'm not sure if we have documented that explicitly enough that users
are not relying on it silently (yes, gnulib, I'm looking at you).

Other header and library checks must mostly be seen as possibly
dependent.  On top of that, you can have arbitrary shell code in between
which implicitly depends on what happened before.

Even if we parallelize all that we can, just the remaining serial bits
from the list of serial things above can already mean that Amdahl won't
give us more than maybe a factor of two speedup.  It might be pretty
disappointing in the end.

As to functionality, we've had mixed success with the Autotest
parallelization, depending on the shell.  I've been contemplating using
GNU parallel where present (if Ole can speed up its startup time).  But
external tools like that or 'make' are pretty much dead in the water
with current Autoconf implementation because we do not have any idea of
what shell variables are needed in the environment for the next tests.
(Not to speak of 32K environment+command line length on MSYS.)

Cheers,
Ralf