Re: [FYI] {testsuite-work} tests: use `$SHELL' to run the shell scripts from `lib/'

[Peter Rosin]

Den 2011-06-02 17:36 skrev Stefano Lattarini:
> This should offer greater testsuite coverage for those developers
> that override CONFIG_SHELL at configure time in order to test more
> shells on a single system, instead of just the default `/bin/sh'.

Do we not want to test these scripts in the same way that they are
later used?  Isn't that more important compared to the convenience
it might be to test things using various shells on a single machine?
If we don't run the scripts with /bin/sh in the testsuite, we might
miss some instance where the script is broken on the "lesser" shell
even though the path taken through the script _should_ not reguire
an xsi-shell, e.g. in the compile2.test case.

I.e. IMHO, it _might_ be ok to do this change for tests that require
an xsi-shell, but otherwise not.  I think it's better to keep skipping
the tests if not using an xsi-shell because the "increased coverage"
argument has flaws:
1. it decreases test coverage for code intended for /bin/sh
2. it is common to run the testsuite with an xsi-shell

Or are you saying that the xsi-shell requirement checks if $SHELL
is an xsi-shell?  In that case we need a new xsi-bin-sh
requirement instead of this patch.

> This change also fixes few spurious failures in tests using the
> `xsi-shell' requirement, where inconsistencies could crop up if
> the shell probed for XSI features (which, by default, is $SHELL)
> was not the same shell later used to run the scripts using those
> features (which was hard-coded to `/bin/sh').  Such failures have
> already occurred in practice, for examples on Solaris systems
> which had also GNU Bash installed.

s/fixes few/fixes a few/
s/for examples/for example/
s/which had also GNU Bash/with GNU Bash/

> * tests/ar-lib.test: Run the `ar-lib' script with `$SHELL', rather
> than directly with `./ar-lib', which would make run unconditionally
> with `/bin/sh'.

s/would make run/makes it run/

> * tests/compile.test: Likewise, but for the `compile' script.
> * tests/compile2.test: Likewise.
> * tests/compile3.test: Likewise.
> * tests/compile4.test: Likewise.
> * tests/compile5.test: Likewise.
> * tests/compile6.test: Likewise.
> * tests/instsh2.test: Likewise, but for the `install' script.
> * tests/instsh3.test: Likewise.
> * tests/mkinst3.test: Likewise, but for the `mkinstalldirs' script.
> * tests/missing.test: Likewise, but for the `missing' script.
> * tests/missing2.test: Likewise.
> * tests/missing3.test: Likewise.
> * tests/missing5.test: Likewise.

*snip*

> diff --git a/tests/compile3.test b/tests/compile3.test
> index f949d1c..141a17a 100755
> --- a/tests/compile3.test
> +++ b/tests/compile3.test
> @@ -30,23 +30,23 @@ END
>  chmod +x ./cl
>  
>  # Check if compile handles "-o foo", -I, -l, -L, -Xlinker -Wl,
> -opts=`LIB= ./compile ./cl foo.c -o foo -lbar -Lgazonk -Ibaz -Xlinker foobar 
> -Wl,-foo,bar`
> +opts=`LIB='' $SHELL compile ./cl foo.c -o foo -lbar -Lgazonk -Ibaz -Xlinker 
> foobar -Wl,-foo,bar`

The LIB='' change is not mentioned under tests/compile3.test in
ChangeLog.  What purpose do the ticks serve anyway?

*snip*

> diff --git a/tests/instsh3.test b/tests/instsh3.test

*snip*

> +
> +:

Not mentioned in ChangeLog.

*snip*

> diff --git a/tests/missing3.test b/tests/missing3.test

*snip*

>  grep WARNING stderr && Exit 1
>  grep Unknown stderr
> +
> +:

Not mentioned in ChangeLog.

*snip*

> diff --git a/tests/missing5.test b/tests/missing5.test
> index 010b344..91e5857 100755
> --- a/tests/missing5.test
> +++ b/tests/missing5.test
> @@ -33,9 +33,9 @@ AC_OUTPUT
>  EOF
>  
>  for tool in $needed_tools; do
> -  cat >$tool.in <<EOF
> -#! /bin/sh
> -exec @$tool@ "\$@"
> +  unindent >$tool.in <<EOF
> +    #! /bin/sh
> +    exec @$tool@ "\$@"

Not mentioned in ChangeLog.

*snip*

> +
> +:

Not mentioned in ChangeLog.

*snip*

> diff --git a/tests/mkinst3.test b/tests/mkinst3.test

*snip*

>  test -d "`pwd`/~a b/-x  y"
>  rm -rf '~a b'
> +
> +:

Not mentioned in ChangeLog.

Now, the million dollar question:  What other changes that were not
mentioned in the ChangeLog did I miss?

Sneaking in a few unrelated changes in a big "scriptable change"-
type patch makes it hard to find the unrelated changes.  It appears
that the above "sneaky changes" that I point out are "just" style
issues?

Cheers,
Peter