more Autotest thoughts

[Ralf Wildenhues]

Continuing on my wishlist of Autotest improvements: (some of this may
sound like a rant, but it is not meant that way, rather a consequence
of the amount of time spent finding some of the stuff out).

It would be nice if autotest put each test (or test group) into a
separate file, and then source or exec it in a subshell.  This is
faster when all you do is `testsuite -v 123'.  Libtool's small test
suite takes noticeable time to skip over 10 tests.  Old shells with
link-farming for here documents will take a very long time because of
the huge case statement.  Is there an m4 limitation that prevents a
fix for this?

Furthermore, the AT_CHECK docs are not fully clear to me: If COMMANDS
fail, but RUN_IF_FAIL is given, then AT_CHECK still only acts upon the
COMMANDS exit status.  In other words: it is not possible to change
the exit status of the test group from within RUN_IF_FAIL (thinking
this may possible may be a flaw of mine -- or suboptimal docs).  Also,
the interaction of COMMANDS exit status and STDOUT and STDERR
expectations are IMHO not described accurately.  Namely, if COMMANDS
exit with 77, but also produce unwanted output, the test fails.  This
is a valid, but unfortunate combination for me, as it means I might
have to discard or ignore output.

AT_DATA doc should include the fact that no shell expansion will take
place on the second argument, but on the first.  I would like a macro
similar to AT_DATA which shell-expands its second argument.  But
frankly, I really don't see the advantage of either command over just
using

| for i in 1 2; do
|   cat >expanded_file$i <<EOF
| shell $expanded $contents.
| EOF
|   cat >unexpanded_file$i <<'EOF'
| not shell $expanded $contents.
| EOF
| done

or even (assuming backslashes are not part of the data),

| for i in 1 2; do
|   echo >expanded_file$i "shell $expanded
| $contents."
|   echo >unexpanded_file$i 'not shell $expanded
| $contents.'
| done

(Libtool even has $echo/$ECHO for stuff with backslashes).
Somebody may please enlighten me.

The apparent beauty of m4 macros seems IMVHO to be traded for increased
output size more often than is technically necessary.

Regards, and yes, I'd love to be convinced to the contrary,
Ralf