[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Cleaner way to run all tests, despite some failing?

From: Jason Pearce
Subject: Re: Cleaner way to run all tests, despite some failing?
Date: Wed, 20 Apr 2005 09:57:15 +0100
User-agent: Mozilla Thunderbird 1.0 (Windows/20041206)

I agree, it always seems a bit ugly to do this in make, or should we say shell. Actually I've totally given up! I am now of the opinion that it is better done elsewhere.

I use a perl script. Each test can be run individually with make
> make test.log
And it actually makes a test.log if it passes and if it fails.

Perl then calls make for each test using its system call - which is now well supported across a number of platforms. Linux, SunOS and Cygwin all work with this approach. Because we still use make to handle dependancies, if a test is already up to date it is not re-run. I then have perl to do more sophisticated control and reporting.
o run a subset of tests based on regular expression matches.
o call make as an argument to time, so I can report on the run time of each test.
o prety print a nice report file


Eric Hanchrow wrote:

Here's the problem:

I have a few executables in the current directory, which are tests.
When I type "make check", I'd like them all to run -- even if some
fail, I'd like the others to run.  And I'd like to collect some status
while they run, which gets reported after the last one finishes.

Here's how I've solved this:

   TESTS := foo bar baz

   check: $(TESTS)
           for t in $(TESTS);                                                   
                   echo -n $$t ...;                                             
                   ./$$t || failed_tests="$$failed_tests $$t";                  
           test -z "$$failed_tests" || { echo Failed tests: $$failed_tests; 
false ; }

   .PHONY: check

This works fine, but it's ... shall we say ... a tad ugly.  (And it
probably subtly depends on features of the shell I happen to be
using.)  Is there a cleaner way to do this?

reply via email to

[Prev in Thread] Current Thread [Next in Thread]