[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: TAP support in automake (was: Re: automake 1.11.3 check-TESTS and co
Re: TAP support in automake (was: Re: automake 1.11.3 check-TESTS and command line length)
Sat, 31 Mar 2012 15:23:49 -0500 (CDT)
Alpine 2.01 (GSO 1266 2009-07-14)
On Sat, 31 Mar 2012, Stefano Lattarini wrote:
Well, automake being automake, it only depends on portable awk ;-)
OK, portable "nawk" actually (as found by the autoconf builtin macro
AC_PROG_AWK), but I don't know of any non-museum machine lacking that.
The new feature is already documented in the automake manual on the
'master' branch; and this documentation should be release-quality
already. But maybe I should make this new feature more visible before
the 1.12 release? If yes, how do you suggest to do so?
Certainly mentioning and discussing it again on the 'automake' list
(happening now) is useful. The previous discussion I found was from
almost a year ago before development of the feature was started.
As for the implementation quality, there's still an annoying glitch in
that you need to *manually* copy the automake-provided 'tap-driver.sh'
script yourself into your package -- "automake --add-missing" won't
work -- and define some variables by hand; all of this is documented
in the manual, and should hopefully be fixed for automake 1.12.1.
This does not seem like a big problem.
Russ Allbery's C TAP Harness looks reasonable as well. It also
requires adding a single file (.c) to the source tree and a rule to
build it as a test program.
The philosophy of TAP is rather different than Automake's existing
test suite. These differences may prove challenging when switching
from Automake tests to TAP:
* Every TAP test needs to somehow know its unique test number (so it
can print a result "ok 35"). If the tests are all in the same
module then they can simply increment an integer but if they are
not then there is a problem (or they can always plan for one
* TAP tests need to know if they are expected to pass or fail.
A major feature of how GraphicsMagick is using the Automake tests
framework is that it determines which tests are XFAIL based on what
features were configured into the software. The list of tests which
are expected to fail is produced based on Automake conditionals. It
is useful for the user to see tests XFAIL as a reminder that the
configuration may be missing something they wanted. How is this best
handled for TAP tests?
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/