Re: [PATCH] {master} tests: more consistent checks about invalid options

[Stefano Lattarini]

On Tuesday 11 January 2011, Ralf Wildenhues wrote:
> Hi Stefano,
> 
> * Stefano Lattarini wrote on Tue, Jan 11, 2011 at 01:48:41PM CET:
> > Another (minor) testsuite patch.
> 
> Discouraged, but OK with nit addressed.
> 
> Thanks,
> Ralf
> 
> > * tests/aclocal.test: Grepping of automake stderr for messages
> > reporting invalid options made stricter.
> > * tests/no-outdir-option.test: Likewise.  Also, create a dummy
> > `Makefile.am', to ensure that the automake failures are really
> > caused only by unrecognized options.
> > * tests/automake.test: Added trailing `:' command.  Removed
> > redundant checks on `--help' and `--version' option (already
> > performed in the test `help*.test').
> 
> > --- a/tests/no-outdir-option.test
> > +++ b/tests/no-outdir-option.test
> 
> > @@ -20,10 +20,12 @@
> >  
> >  set -e
> >  
> > +: > Makefile.am
> > +
> >  AUTOMAKE_fails -Wno-error --output-dir=foo
> > -$EGREP '(invalid|unrecognized) option.*--output-dir' stderr
> > +grep 'unrecognized option.*--output-dir' stderr
> 
> This strikes me as wrong.  Why would you have written the test like this
> in the first place if there wasn't a good reason for this?
> (And just the time required to think about this, track down the reason
> for the original choice, etc., wastes more than would have been gained
> by this patch IMVHO.)
>
> Maybe because Getopt::Long::GetOptions could throw a (differently
> spelled) error itself?
>
That's what I was suspecting myself when I wrote this hunk; so, instead
of risking some spurious failure in later on-field testing, I preferred
to be a bit lax in the grepping of the error message (after all, both
"invalid option" and "unrecognized option" are good and clear messages).

But then I saw that the similar tests aclocal.test and automake.test
are stricter in their grepping of error messages, and no failure has
been reported against them as of today -- so I realized that such a
laxness was uncalled for.  And being more consistent wouldn't hurt,
either.

And this patch was borne.  Does this explanation clarifies things?

Thanks,
  Stefano