automake
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: running tests under in a tests/ directory


From: aaragon
Subject: Re: running tests under in a tests/ directory
Date: Fri, 5 Jun 2009 12:05:18 -0700 (PDT)

Hi Ralf,


Ralf Wildenhues wrote:
> 
> * aaragon wrote on Wed, Jun 03, 2009 at 09:22:59PM CEST:
>> Ralf Wildenhues wrote:
>> > * aaragon wrote on Sat, May 30, 2009 at 07:34:13PM CEST:
>> >> check:        all
>> >>       @echo "Running tests"
>> >>       cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s test
> 
>> >> Except that if I change 'check: all' to something else, then it
>> >> doesn't work anymore.
>> > 
>> > "doesn't work"  is too vague.  What does not work, what error message
>> do
>> > you get, more details please?
>> 
>> In your first reply you suggested to use check-local, but if I put in the
>> Makefile.am in the root director:
>> 
>> check:       check-local
> 
> Ah, that's not what I meant, sorry for being imprecise.  I meant to
> use
> 
> check-local:
>         @echo ...
> 
> 

I got it now, I replaced it by:

check-local:
        @echo 
"-------------------------------------------------------------------"
        @echo "All tests have been run. Write additional tests under the tests"
        @echo "directory and add the test names to the TESTEXECS variable under"
        @echo "tests/Makefile.am"
        @echo 
"-------------------------------------------------------------------"



Ralf Wildenhues wrote:
> 
> 
> IOW, replace 'check' with 'check-local'.  And then, because 'check' will
> invoke 'check-local', and 'check' will recurse already anyway, you can
> omit the recursion and just let it invoke the check-am target in
> tests/Makefile.
> 
>> >> Now, from the code that I got from the internet, I really liked the
>> >> fact that you don't need to specify the name of the tests in the
>> >> Makefile.am file because of the use of wildcards. Is there a way to
>> >> make something similar here that is really portable?
>> > 
>> > Not really; well, at the very least you will need to have a naming
>> > scheme such that all tests have some specified suffix(es), for example
>> > '.test'.  Your sources would then match '*.test.cpp'.
> 
>> Can you be more specific on this? All my files start with the word
>> 'test',
>> but I can change the file names so that they end in 'test.cpp' if needed.
>> However, I tried to put 
>> 
>> TESTEXECS = 'test*.cpp'
>> 
>> in the Makefile.am under the tests directory and it gives me an error:
> 
> Yeah, currently it is not possible to use wild cards there.  Sorry for
> giving that false impression, I didn't think it fully through.
> 
> 

That's a pity, I hope in the future there something that can fix this.


Ralf Wildenhues wrote:
> 
> 
>> Ralf Wildenhues wrote:
>> >> If I make one of the tests fail (by deliberately changing one of the
>> >> verified files), I get a message saying that a binary operator is
>> >> expected:
>> >> 
>> >> address@hidden/Documents/workspace/cpputils/tests$make check
>> >> make  check-TESTS
>> >> FAIL: test001
>> >> PASS: testcppblas
>> >> /bin/sh: line 0: test: Alejandro: binary operator expected
>> >> ========================
>> >> 1 of 2 tests failed
>> >> See tests/test-suite.log
>> >> ========================
>> >> make[2]: *** [test-suite.log] Error 1
>> >> make[1]: *** [check-TESTS] Error 2
>> >> make: *** [check-am] Error 2
>> >> 
>> >> Is this normal?
> 
> 

When deliberately making the tests fail, I still get:

address@hidden/Documents/workspace/cpputils$make check
Making check in cpputils
make[1]: Nothing to be done for `check'.
Making check in tests
make  check-TESTS
FAIL: test001
PASS: testcppblas
==============================================================
1 of 2 tests failed
See tests/test-suite.log
Please report to Alejandro Aragon <address@hidden>
==============================================================
make[3]: *** [test-suite.log] Error 1
make[2]: *** [check-TESTS] Error 2
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1

and I removed the double quoting from the AC_INIT macro already.


Ralf Wildenhues wrote:
> 
> 
>> > 
>> > Not sure, it could be a bug in my suggested code or in Automake.  Can
>> > you post the output of
>> > 
>> >   make check SHELL=/bin/sh\ -x
>> > 
>> > for that, as well as the tests/Makefile.am and the automake version
>> > you're using?  Thanks.
> 
>> I'm using:
>> address@hidden/Documents/workspace/cpputils$automake --version
>> automake (GNU automake) 1.11
> 
>> If I type 'make check' and the tests are successful, I get the following
>> error at the end:
>> 
>> address@hidden/Documents/workspace/cpputils$make check
> 
>> -------------------------------------------------------------------
>> Running tests
>> cd tests && make  -s test
>> make[1]: *** No rule to make target `test'.  Stop.
>> make: *** [check] Error 2
>> address@hidden/Documents/workspace/cpputils$
>> 
>> 
>> I think that error is because of the following line in the Makefile.am in
>> the root director:
>> 
>>      cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s test
>> 
>> Shouldn't I just use
>> 
>>      cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s
>> 
>> ?
> 
> Well, if there is no "test" rule defined in tests/Makefile.am, then it
> cannot be run.  But automake already provides recursion for you, with
> the check rule, so you can just use that and need not build your own.
> See above.
> 
>> Here I make one of the tests fail deliberately. This is the output of
>> "make
>> check SHELL=/bin/sh\ -x" for this case, where you can see the error I was
>> referring to in my post at the end:
>> 
>> address@hidden/Documents/workspace/cpputils$make check SHELL=/bin/sh\ -x
>> + test '!' -f config.h
> 
>> + msg='1 of 2 tests failed.  See tests/test-suite.log.  '
>> + test -n Alejandro 'Aragon <address@hidden>'
>> /bin/sh: line 0: test: Alejandro: binary operator expected
> 
> This indicates that your PACKAGE_BUGREPORT setting from configure.ac
> contains weird kind of quoting, namely both double and single quotes.
> Can you omit the single quotes?
> 
> Cheers,
> Ralf
> 
> 


One more comment. You forgot to add the diff-driver file to the EXTRA_DIST
variable.

Regards,

Alejandro
-- 
View this message in context: 
http://www.nabble.com/running-tests-under-in-a-tests--directory-tp23773808p23893836.html
Sent from the Gnu - Automake - General mailing list archive at Nabble.com.





reply via email to

[Prev in Thread] Current Thread [Next in Thread]