[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#20715: tap-driver.sh 'missing test plan' message contradicts the TAP
From: |
Nick Bowler |
Subject: |
bug#20715: tap-driver.sh 'missing test plan' message contradicts the TAP Standard |
Date: |
Wed, 3 Jun 2015 10:25:13 -0400 |
User-agent: |
Mutt/1.5.23 (2014-03-12) |
On 2015-06-02 11:08 -0700, Arthur Schwarz wrote:
> [Nick Bowler wrote]
> > On 2015-06-02 09:40 -0700, Arthur Schwarz wrote:
> > > The TAP Standard specifically states that if the test plan is 1..N
> > > and the number of test lines are k < N, then the k+1 .. N missing
> > > test lines are to be considered as skipped.
> >
> > Where does it say that? It is not correct to treat missing test results
> > as skips.
>
> From http://testanything.org/tap-version-13-specification.html
[...]
> The following listing plans on running 5 tests. However, our program
> decided to not run tests 2 thru 5 at all. To properly report this, the
> tests are marked as being skipped.
>
> TAP version 13
> 1..5
> ok 1 - approved operating system
> # $^0 is solaris
> ok 2 - # SKIP no /sys directory
> ok 3 - # SKIP no /sys directory
> ok 4 - # SKIP no /sys directory
> ok 5 - # SKIP no /sys directory
This example has exactly 5 test lines, which matches the plan of 5.
So there is no error.
[...]
> TAP allows k < N (see above).
It does not.
Regards,
--
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)