Re: running tests under in a tests/ directory

[Ralf Wildenhues]

Hello Alejandro,

* aaragon wrote on Sat, May 30, 2009 at 07:34:13PM CEST:
> 
> Thank yo Ralf for replying to my post. The code that I showed in my post was
> indeed taken from that website. The code you provided me is way too advanced
> for me, since I consider myself just a user. The main idea that I got from
> your reply is that this new code won't work with older versions of Automake.

Right.  But as William already replied, you only need Automake on your
development system and not on systems where the code is configured and
built.

> Now, regarding the code that you wrote, I am trying to make it work, so I
> changed the Makefile.am in the root directory to what you suggested
> (almost):
> 
> # test rule
> TSTDIR = tests
> 
> check:        all
>       @echo 
> "-------------------------------------------------------------------"
>       @echo "Running tests"
>       cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s test
>       @echo 
> "-------------------------------------------------------------------"
> 
> Except that if I change 'check: all' to something else, then it doesn't work
> anymore.

"doesn't work"  is too vague.  What does not work, what error message do
you get, more details please?

> In the tests directory, I changed the Makefile.am to reflect that I use only
> cpp files. I also made the generate-verified-files silent, so that the
> output of that target is:

> Now, from the code that I got from the internet, I really liked the fact
> that you don't need to specify the name of the tests in the Makefile.am file
> because of the use of wildcards. Is there a way to make something similar
> here that is really portable?

Not really; well, at the very least you will need to have a naming
scheme such that all tests have some specified suffix(es), for example
'.test'.  Your sources would then match '*.test.cpp'.

> If I make one of the tests fail (by deliberately changing one of the
> verified files), I get a message saying that a binary operator is expected:
> 
> address@hidden/Documents/workspace/cpputils/tests$make check
> make  check-TESTS
> FAIL: test001
> PASS: testcppblas
> /bin/sh: line 0: test: Alejandro: binary operator expected
> ========================
> 1 of 2 tests failed
> See tests/test-suite.log
> ========================
> make[2]: *** [test-suite.log] Error 1
> make[1]: *** [check-TESTS] Error 2
> make: *** [check-am] Error 2
> 
> Is this normal?

Not sure, it could be a bug in my suggested code or in Automake.  Can
you post the output of

  make check SHELL=/bin/sh\ -x

for that, as well as the tests/Makefile.am and the automake version
you're using?  Thanks.

Cheers,
Ralf