[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: testing exceptions + bug?
Re: testing exceptions + bug?
Fri, 23 Feb 2001 12:20:11 +0100 (MET)
> my first commit is test-suite/tests/exceptions.test, which was inspired
> by the (cond (1 => (lambda (x y) y))) problem reported earlier this
> week. that case is now working, but in writing other tests, i think
> i've found at least one other bug. the failing tests are marked "no
> exception" in the code, and are revealed when doing a "make check".
Welcome to the club :-), and thanks a lot for providing tests. However, I
would like to comment a bit on your tests:
I think that the file name exceptions.test is unfortunate: You are
actually testing the reader and implementation of syntactic forms here,
not the execption code. Following your paradigm, then _all_ tests that
check for correct parameter checking etc. would have to go into
exceptions.test, because all these tests except exceptions to be thrown,
but this obviously does not make sense.
It would be a good idea, however, to look through the test files and to
extract common patterns into a library that helps to write test cases
conveniently. Your expect-exception macro is a candidate for such a
library. However, if put into a library, expect-exception should not make
the test code part of the test name. If the code is longer than '(lambda
(x x x) 1)' this can become disturbing.
Further, we should try not to obfuscate test code. Test code has to be
read by people as other code as well. I have spent some time to figure
out whether your 'rubbish' definitions in the big W were actually used :-(
Further, I would prefer an explicit name like expect-exception over goad.
Test code in guile has not been taken very seriously, but this is not a
culture that we should be proud of, and neither should we continue that
> please let me know if i've misunderstood something, or if it's better to
> not commit tests that cause "make check" to fail, etc. (i usually frown
> on nightly-build breakage, like this.)
It is always good to commit tests, even if they fail. For this purpose,
however, there is the 'expect-fail' form available as an alternative for
'pass-if'. expect-fail indicates that we know about a problem, but that
it is not solved yet. In your case, it would make sense to have a
corresponding expect-fail-exception in contrast to expect-exception.
My suggestion would be to define pass-if-exception and
expect-fail-exception in lib.scm (at the bottom, replacing the
signals-error stuff). Both should be defined similar to pass-if and
(pass-if-exception name exception body) and
(expect-fail-exception name exception body)
By using names like pass-if* and expect-fail* it should be easier to grep
for tests that are expected to fail in the test suite.
Sorry for ranting. I very much appreciate you approach to provide tests
in general. And, I think with respect to writing tests we have not yet
had a thorough discussion. Thus, others may have different opinions about
how the test suite should look like. Maybe we can bring up a discussion
on the list?