[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: running tests under in a tests/ directory
From: |
aaragon |
Subject: |
Re: running tests under in a tests/ directory |
Date: |
Fri, 5 Jun 2009 12:05:18 -0700 (PDT) |
Hi Ralf,
Ralf Wildenhues wrote:
>
> * aaragon wrote on Wed, Jun 03, 2009 at 09:22:59PM CEST:
>> Ralf Wildenhues wrote:
>> > * aaragon wrote on Sat, May 30, 2009 at 07:34:13PM CEST:
>> >> check: all
>> >> @echo "Running tests"
>> >> cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s test
>
>> >> Except that if I change 'check: all' to something else, then it
>> >> doesn't work anymore.
>> >
>> > "doesn't work" is too vague. What does not work, what error message
>> do
>> > you get, more details please?
>>
>> In your first reply you suggested to use check-local, but if I put in the
>> Makefile.am in the root director:
>>
>> check: check-local
>
> Ah, that's not what I meant, sorry for being imprecise. I meant to
> use
>
> check-local:
> @echo ...
>
>
I got it now, I replaced it by:
check-local:
@echo
"-------------------------------------------------------------------"
@echo "All tests have been run. Write additional tests under the tests"
@echo "directory and add the test names to the TESTEXECS variable under"
@echo "tests/Makefile.am"
@echo
"-------------------------------------------------------------------"
Ralf Wildenhues wrote:
>
>
> IOW, replace 'check' with 'check-local'. And then, because 'check' will
> invoke 'check-local', and 'check' will recurse already anyway, you can
> omit the recursion and just let it invoke the check-am target in
> tests/Makefile.
>
>> >> Now, from the code that I got from the internet, I really liked the
>> >> fact that you don't need to specify the name of the tests in the
>> >> Makefile.am file because of the use of wildcards. Is there a way to
>> >> make something similar here that is really portable?
>> >
>> > Not really; well, at the very least you will need to have a naming
>> > scheme such that all tests have some specified suffix(es), for example
>> > '.test'. Your sources would then match '*.test.cpp'.
>
>> Can you be more specific on this? All my files start with the word
>> 'test',
>> but I can change the file names so that they end in 'test.cpp' if needed.
>> However, I tried to put
>>
>> TESTEXECS = 'test*.cpp'
>>
>> in the Makefile.am under the tests directory and it gives me an error:
>
> Yeah, currently it is not possible to use wild cards there. Sorry for
> giving that false impression, I didn't think it fully through.
>
>
That's a pity, I hope in the future there something that can fix this.
Ralf Wildenhues wrote:
>
>
>> Ralf Wildenhues wrote:
>> >> If I make one of the tests fail (by deliberately changing one of the
>> >> verified files), I get a message saying that a binary operator is
>> >> expected:
>> >>
>> >> address@hidden/Documents/workspace/cpputils/tests$make check
>> >> make check-TESTS
>> >> FAIL: test001
>> >> PASS: testcppblas
>> >> /bin/sh: line 0: test: Alejandro: binary operator expected
>> >> ========================
>> >> 1 of 2 tests failed
>> >> See tests/test-suite.log
>> >> ========================
>> >> make[2]: *** [test-suite.log] Error 1
>> >> make[1]: *** [check-TESTS] Error 2
>> >> make: *** [check-am] Error 2
>> >>
>> >> Is this normal?
>
>
When deliberately making the tests fail, I still get:
address@hidden/Documents/workspace/cpputils$make check
Making check in cpputils
make[1]: Nothing to be done for `check'.
Making check in tests
make check-TESTS
FAIL: test001
PASS: testcppblas
==============================================================
1 of 2 tests failed
See tests/test-suite.log
Please report to Alejandro Aragon <address@hidden>
==============================================================
make[3]: *** [test-suite.log] Error 1
make[2]: *** [check-TESTS] Error 2
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1
and I removed the double quoting from the AC_INIT macro already.
Ralf Wildenhues wrote:
>
>
>> >
>> > Not sure, it could be a bug in my suggested code or in Automake. Can
>> > you post the output of
>> >
>> > make check SHELL=/bin/sh\ -x
>> >
>> > for that, as well as the tests/Makefile.am and the automake version
>> > you're using? Thanks.
>
>> I'm using:
>> address@hidden/Documents/workspace/cpputils$automake --version
>> automake (GNU automake) 1.11
>
>> If I type 'make check' and the tests are successful, I get the following
>> error at the end:
>>
>> address@hidden/Documents/workspace/cpputils$make check
>
>> -------------------------------------------------------------------
>> Running tests
>> cd tests && make -s test
>> make[1]: *** No rule to make target `test'. Stop.
>> make: *** [check] Error 2
>> address@hidden/Documents/workspace/cpputils$
>>
>>
>> I think that error is because of the following line in the Makefile.am in
>> the root director:
>>
>> cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s test
>>
>> Shouldn't I just use
>>
>> cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s
>>
>> ?
>
> Well, if there is no "test" rule defined in tests/Makefile.am, then it
> cannot be run. But automake already provides recursion for you, with
> the check rule, so you can just use that and need not build your own.
> See above.
>
>> Here I make one of the tests fail deliberately. This is the output of
>> "make
>> check SHELL=/bin/sh\ -x" for this case, where you can see the error I was
>> referring to in my post at the end:
>>
>> address@hidden/Documents/workspace/cpputils$make check SHELL=/bin/sh\ -x
>> + test '!' -f config.h
>
>> + msg='1 of 2 tests failed. See tests/test-suite.log. '
>> + test -n Alejandro 'Aragon <address@hidden>'
>> /bin/sh: line 0: test: Alejandro: binary operator expected
>
> This indicates that your PACKAGE_BUGREPORT setting from configure.ac
> contains weird kind of quoting, namely both double and single quotes.
> Can you omit the single quotes?
>
> Cheers,
> Ralf
>
>
One more comment. You forgot to add the diff-driver file to the EXTRA_DIST
variable.
Regards,
Alejandro
--
View this message in context:
http://www.nabble.com/running-tests-under-in-a-tests--directory-tp23773808p23893836.html
Sent from the Gnu - Automake - General mailing list archive at Nabble.com.
- Re: running tests under in a tests/ directory, William Pursell, 2009/06/01
- Re: running tests under in a tests/ directory, Ralf Wildenhues, 2009/06/03
- Re: running tests under in a tests/ directory, aaragon, 2009/06/03
- Re: running tests under in a tests/ directory, Ralf Wildenhues, 2009/06/04
- Re: running tests under in a tests/ directory,
aaragon <=
- Re: running tests under in a tests/ directory, Ralf Wildenhues, 2009/06/06
- Re: running tests under in a tests/ directory, aaragon, 2009/06/09
- Re: running tests under in a tests/ directory, aaragon, 2009/06/09
- Re: running tests under in a tests/ directory, Bob Friesenhahn, 2009/06/09
- Re: running tests under in a tests/ directory, Ralf Wildenhues, 2009/06/09
- Re: running tests under in a tests/ directory, aaragon, 2009/06/09
- Re: running tests under in a tests/ directory, Bob Friesenhahn, 2009/06/09
- Re: running tests under in a tests/ directory, Peter Johansson, 2009/06/09
- Re: running tests under in a tests/ directory, Bob Friesenhahn, 2009/06/09
- Re: running tests under in a tests/ directory, Peter Johansson, 2009/06/09