Re: AC_REQUIRE problems

[Dan Manthey]

On Tue, 15 Feb 2005, Stepan Kasal wrote:
> On Mon, Feb 14, 2005 at 10:25:56AM -0800, Paul Eggert wrote:
> > Fair enough.  But why not support parallelism while you're at it?
> > This can be done in the shell, albeit less elegantly than in 'make'.
        I'm not impressed with the idea of parallelism in configuration.
Broadly speaking, the act of configuration is that of setting variables to
describe the build environment.  Necessarily, every test that occurs must
set at least one variable (possibly more), and that set operation must
have approiate locking.  Make avoids this issue by running all its
commands in subshells where they _can't_ set variables (making Make pretty
inconvenient for our purposes).
        An a gedanken experiment, imagine if we try to put parellelism in
the shell.  We would need to run tests in asynchronous subshells.  The
results of those subshells would need to be returned to the main shell
process (the one that holds all the variables that get AC_SUBST'ed).
Asynchronous processes really don't have very many ways of returning
values to their callers: exit statuses (barely better than a yes/no value,
since I'm pretty sure non-zero exit statuses are very non-portable), or
the contents of the standard output.  I'm ignorant of how the latter can
even be accessed asynchronously.  Now, it is possible (though ugly) for
such an output to do things like set variables by passing it to `eval',
but this seems complicated, and still requires appropirate locking.
        Furthermore, asynchronous processes are only truly asynchronous if
the caller of them does two separate things: first, the asynch process
must be requested, and then _later_ used.  If there is no way to request a
test before it is needed, then there is no benefit to running it
asynchronously.  So, if we move (say via a diversion) start_test() well
before wait_for_test(), we get

        start_test()
        # do other stuff, like starting other tests.
        if wait_for_test(); then
          foo
        fi

but this fails if the test is only sometimes needed:

        start_test_1()
        start_test_2()  # INVALID if test_1 fails.
        # other stuff
        if wait_for_test_1(); then
          # more stuff
          if wait_for_test_2(); then
            bar
          fi
        fi

at which point we get:

        start_test_1()
        # other stuff
        if wait_for_test_1(); then
          start_test_2()
          # more stuff
          if wait_for_test_2(); then
            bar
          fi
        fi

I think in the usual usage, `# other stuff' is small or non-existant,
making the parrellelism almost non-existant and probably not worth the
effort.  I conclude that you really can't use parellelism without having a
strictly declarative language, which would, as Stepan notes, require a
whole new project, not just a change to Autoconf.

> I think that adding parallelism might bring additional problems, which
> could be hard to debug.  For example: make has a limit on the number of
> processes; in shell, you probably have to count the number of processes
> yourself.
        I know that make goes through some terrible contortions to get
this right.  *Grin* Then again, my cynical view is that even without any
limits, there'd be close to no parallelism.

> Well, on a second thought, you are probably right, paralellism in the
> next thing to do.  I just don't plan to do it.
        I think that parallelism is a bad goal.  The only reason we're
discussing in now anyway is that it comes along with using Make for
dependencies.  I don't think Make is the right solution to that either;
Make does a good job of repeatedly using the same dependency graph with
different states of freshness.  This requires that the graph
be known independent of the result of each node.  Configuration rarely
changes the freshness of nodes (You're compiler doesn't suddenly start
supporting `inline' because you found out that it has a particular
header.) but usually builds the graph on the basis of the result of
earlier nodes (You only need to check for some header after its preferred
alternative has not been found.).
        As more pragmatic arguments against Make:  (1) It runs commands in
subshells, making it a pain to actually record results (other than
timestamps).  (2) Have you ever tried putting a here document in Make?  I
found it impossible last time I tried.

-Dan