bug#20715: tap-driver.sh 'missing test plan' message contradicts the TAP Standard

[Arthur Schwarz]

The issue is not intent but information. The Automake Manual says that their
TAP implementation is defined by the TAP references. The reference to the
TAP Standard is missing which is puzzling, and the remaining references can
not be considered definitive but only an instance of an implementation of
the standard. The 'fault' is that in the face of conflicting statements in
the standard, Automake does not declare which is to be used. No biggie
unless you plan to use TAP.

Here is an alternative description of the TAP area of the Automake document.
It is unproofed, unfinished, and inelegant. It is 'free' in the sense that I
have lost all my senses and assigned my GPL copyright to the FSF. Sigh.

art

-----Original Message-----
From: Nick Bowler [mailto:address@hidden 
Sent: Tuesday, June 02, 2015 6:41 AM
To: Arthur Schwarz
Cc: address@hidden
Subject: Re: bug#20715: tap-driver.sh 'missing test plan' message
contradicts the TAP Standard

On 2015-06-01 14:45 -0700, Arthur Schwarz wrote:
> http://testanything.org/
> 
> Explicitly specifies that a test plan is optional.
[...]
> The operand part of this paragraph is " The plan is optional ..." which
> directly contradicts "It must appear once ...". Your diagnostic message is
> either incorrect, or more likely, there should be a statement in the
> Automake Manual providing guidance.

The plan line has to be mandatory for this feature to work as
intended (i.e., for the TAP consumer to determine whether a
producer has run to completion or not).  An optional plan would
be useless.

Other TAP consumers behave similarly to Automake's in this regard.
Take prove for example, the de-facto reference implementation of a
TAP consumer, which outputs a similar error message:

  % prove -ve sh test.sh
  test.sh .. 
  ok 1
  All 1 subtests passed 

  Test Summary Report
  -------------------
  test.sh (Wstat: 0 Tests: 1 Failed: 0)
    Parse errors: No plan found in TAP output
  Files=1, Tests=1,  0 wallclock secs ( 0.03 usr +  0.00 sys =  0.03 CPU)
  Result: FAIL

The bug is in the TAP documentation.

Cheers,
-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

TAP.pdf
Description: Adobe PDF document