potential bash-specific bug in `tests/defs.in'

[Stefano Lattarini]

Hi everybody.

In these days, I've been playing with the automake source code
(just to "instruct" myself by seeing how a real-word, working
sotfware is done), and I run into what (I think) can be described
as a potential bug in the testsuite, specifically related to the
exit trap set up in the file `tests/defs.in'.

In fact, if I create a dummy testcase script with the `set -e' flag
active and containing a syntax error, and run it using the bash shell
(either version 3.2 or 4.0), the test case succeeds, since a `0'
exit status is erroneusly passed to the code in the exit trap.

This bug shouldn't manifest itself frequently (after all, how often
does someone write a testcases containing an unseen syntax
error?), but can be potentially harmful, making a broken test
to be considered as passed.


These steps should reproduce the bug:
 $ git clone git://git.savannah.gnu.org/automake.git
 $ cd automake
 $ ./bootstrap
 $ ./configure
 $ cd tests
 $ cat >foo.test <<EOF
 . ./defs || exit 100
 set -e
 fi # syntax error here
 Exit 1
 EOF
 $ bash foo.test; echo "RC: $?" # bash version 3.2 or 4.0: BAD
 === Running test foo.test
 ++ pwd
 /tmp/automake/tests/foo.dir
 + set -e
 foo.test: line 3: syntax error near unexpected token `fi'
 ++ exit_status=0
 ++ set +e
 ++ cd /tmp/automake/tests
 ++ case $exit_status,$keep_testdirs in
 ++ find foo.dir -type d '!' -perm -200 -exec chmod u+w '{}' ';'
 ++ rm -rf foo.dir
 ++ test 0 '!=' 0
 ++ echo ': exit 0'
 : exit 0
 ++ exit 0
 RC: 0
 
 Other shells behave correctly:

 $ dash foo.test; echo "RC: $?" # dash version 0.5: OK
 === Running test foo.test
 + pwd
 /tmp/automake/tests/foo.dir
 + set -e
 foo.test: 3: Syntax error: "fi" unexpected
 + exit_status=2
 + set +e
 + cd /tmp/automake/tests
 + test 0 != 0
 + echo : exit 2
 : exit 2
 + exit 2
 RC: 2

 $ zsh foo.test; echo "RC: $?" # zsh version 4.3: OK
 === Running test ./defs
 +./defs:427> pwd
 /tmp/automake/tests/defs.dir
 +foo.test:2> set -e
 foo.test:3: parse error near `fi'
 +foo.test:4> exit_status=1
 +foo.test:4> set +e
 +foo.test:4> cd /tmp/automake/tests
 +foo.test:4> case 1, (0,)
 +foo.test:4> test 0 '!=' 0
 +foo.test:4> echo ': exit 1'
 : exit 1
 +foo.test:4> exit 1
 RC: 1


I'm aware that, strictly speaking, this is a bug of bash, but
since the automake testsuite try to use "portable shell code"
only, I think it would be good if it works around that bug,
as it does with bugs and limitations of other shells.

Regards,
    Stefano