Re: Test suite and autotest

[Paul Smith]

On Mon, 2012-03-05 at 21:11 +0000, Tim Landscheidt wrote:
> What is the sentiment concerning converting the test suite
> to autotest?  Attached is an initial, unpolished patch for
> one test.

Hi Tim; thanks for taking an interest in this!

I have concerns about autotest, though I'm not hugely familiar with it.
Some of the things we need to do with the make test suite, I'm not sure
how to do  well in autotest.  For example many messages (especially
errors) that need to be tested by the test suite include values such as
the name of the make program, the pathname to the current directory,
etc. none of which can be made static.  The test suite (check
run_make_tests.pl:run_make_test()) replaces tokens in the makefile
output to get the right content.

Also there are more advanced uses like the ability to run the suite
under valgrind (or other tools in the past but these days I only really
use valgrind, myself).

And to be honest, I like the format of the run_make_test() function much
better than the m4-based format of autotest, although perhaps with some
macro usage that could be fixed up.

Still, autotest does provide some nice features including a framework
with logging etc. and solving the issue of "leaking" information between
test groups etc.  And, it doesn't require Perl which could be a help (or
maybe not: is it more portable to require Perl on non-POSIX-based
systems like Windows, or to require a Bourne shell?)

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <address@hidden>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.mad-scientist.net
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist