Re: coreutils-6.11: misc tests are PATH dependent

[Jim Meyering]

Bruno Haible <address@hidden> wrote:
> After your fix in
>   http://lists.gnu.org/archive/html/bug-coreutils/2008-02/msg00083.html
> "make check" does not fail any more when PATH contains ".", but it still
> produces a bunch of unnecessary output.
>
> Tested with coreutils-6.11 on Linux/x86:
>
> In the tests/misc/ directory, without "." in PATH, the "make check" output is:

[well-behaved]

> When PATH contains "." ahead of the usual */bin directories, the "make check"
> output is:
>
> ===============================================================================
> make[5]: Entering directory `/build/coreutils-6.11/tests/misc'
> fail-1...
> fail-2...

[ill-behaved]
[2000 useless lines elided]

> fail-b...
> fail-c...
> /bin/sh: line 31: test: expr (GNU coreutils) 6.10
> Copyright (C) 2008 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> Written by Mike Parker.: integer expression expected
>
> ========================
>  All 85 tests passed
>  (8 tests were not run)
> ========================
>
> make[5]: Leaving directory `/build/coreutils-6.11/tests/misc'
> ===============================================================================
>
> It appears that
>   1) For each *.log test, the 'basename' test is run.
>   2) After all tests, the 'expr' test is run. It fails with weird error
>      messages, but the final result is still "All 85 tests passed".

Hi Bruno,

Thanks for testing and tracking down the cause.  To summarize, if you have
an unsafe PATH, coreutils' "make check" produces a lot of useless output.
This is because while Makefile.am takes care to set PATH for the actual
test script, some of the test-managing infrastructure (which runs basename
and expr) is run outside of that scope.  coreutils-6.11 has test scripts
in tests/misc/ named basename and expr, and if you put "." too early in
PATH, those commands malfunction.

BTW, those thousands of lines are remarkably similar to what you
reported here for MacOS X 10.5:
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/13270/focus=13273
minus the failures, of course.  So I guess you ran those tests with
"." early in your PATH, too.

If security isn't enough of an argument, you can consider this yet another
reason not to put "." early in your PATH.  Please consider removing
"." from your PATH altogether.  Yes, that does make for some small amount
of extra typing (you have to prefix certain commands with "./"), but
that is a small price to pay for the reduced risk of mishap.
[Sorry to harp on this again, but I wouldn't want readers to get the
 impression that it's ok to have "." *anywhere* in PATH, much less
 near the beginning. ]

However, I have a big patch (nearly complete) that revamps the way tests
are run, and I've just confirmed that with it, "make check" works fine,
even with an abuse-inviting $PATH.