From 486f8b31f351795612847e270d44a254e07bf318 Mon Sep 17 00:00:00 2001 From: Stefano Lattarini Date: Wed, 9 Sep 2009 21:49:03 +0200 Subject: [PATCH] Fix testsuite: avoid Zsh-related problems * tests/README: Describe a bug (or at least a weird behaviour) in the way the 4.x versions of Zsh deal with the `-x' a.k.a. `xtrace' switch. This bug can unexpectedly break apparently flawless test scripts. Describe also a workaround for it, involving the use of the new shell function `run_CMD()', defined in tests/defs.in (see below). * tests/defs.in: New subroutine `run_CMD()'. Implementation of subroutine `AUTOMAKE_run()' rewritten to use `run_CMD()'. Unset variable `TEST_LOG_COMPILER', that might cause spurious failures by leaking in the environment of the make processes executed by the test scripts. Updated the code which ensures Bourne-compatibility in Zsh, by adding a call to `setopt NO_GLOB_SUBST' (as done by autoconf 2.64). * tests/acloca14.test: * tests/acloca17.test: * tests/acloca18.test: * tests/aclocal.test: * tests/aclocal3.test: * tests/aclocal8.test: * tests/acsilent.test: * tests/alpha.test: * tests/ammissing.test: * tests/ansi3.test: * tests/ansi3b.test: * tests/check8.test: * tests/cond39.test: * tests/defs.in: * tests/depcomp2.test: * tests/depcomp4.test: * tests/depend5.test: * tests/depend6.test: * tests/discover.test: * tests/distcom3.test: * tests/fn99.test: * tests/fn99subdir.test: * tests/gnits2.test: * tests/gnits3.test: * tests/help.test: * tests/init.test: * tests/instsh.test: * tests/ldadd.test: * tests/libtool7.test: * tests/libtool9.test: * tests/lisp3.test: * tests/ltorder.test: * tests/man4.test: * tests/mdate.test: * tests/missing3.test: * tests/missing4.test: * tests/missing6.test: * tests/obsolete.test: * tests/output-order.test: * tests/pr220.test: * tests/pr300-ltlib.test: * tests/python11.test: * tests/python4.test: * tests/python5.test: * tests/python6.test: * tests/python7.test: * tests/python8.test: * tests/python9.test: * tests/sanity.test: * tests/subobj.test: * tests/subobj9.test: * tests/unused.test: * tests/vars3.test: * tests/version8.test: use new subroutine `run_CMD()' instead of hand-crafted stdout and/or stderr redirections; this, togheter with the changes in file `tests/defs.in', fixes misbehaviour w.r.t. zsh for (at least) the following tests: - aclocal8.test - acsilent.test - depcomp2.test - getopt.test - location.test - missing3.test - output-order.test - parallel-tests3.test - parallel-tests4.test - unused.test - warnopts.test - check9-p.test --- ChangeLog | 88 ++++++++++++++++++++++++++++++++++++++ tests/README | 23 ++++++++++ tests/acloca14.test | 2 +- tests/acloca17.test | 4 +- tests/acloca18.test | 8 +-- tests/aclocal.test | 6 +- tests/aclocal3.test | 3 +- tests/aclocal8.test | 5 +- tests/acsilent.test | 5 +- tests/alpha.test | 3 +- tests/ammissing.test | 3 +- tests/ansi3.test | 3 +- tests/ansi3b.test | 3 +- tests/check8.test | 8 +--- tests/cond39.test | 6 +-- tests/defs.in | 107 ++++++++++++++++++++++++++++++++++++++++------- tests/depcomp2.test | 7 ++- tests/depcomp4.test | 5 +- tests/depend5.test | 5 +-- tests/depend6.test | 7 +-- tests/discover.test | 3 +- tests/distcom3.test | 5 +- tests/fn99.test | 3 +- tests/fn99subdir.test | 3 +- tests/gnits2.test | 4 +- tests/gnits3.test | 4 +- tests/help.test | 2 +- tests/init.test | 8 +++- tests/instsh.test | 5 +- tests/ldadd.test | 3 +- tests/libtool7.test | 21 ++++----- tests/libtool9.test | 7 ++- tests/lisp3.test | 5 +- tests/ltorder.test | 3 +- tests/man4.test | 6 +-- tests/mdate.test | 3 +- tests/missing3.test | 16 ++++---- tests/missing4.test | 6 +-- tests/missing6.test | 5 +- tests/obsolete.test | 4 +- tests/output-order.test | 9 ++-- tests/pr220.test | 3 +- tests/pr300-ltlib.test | 6 +-- tests/python11.test | 4 +- tests/python4.test | 3 +- tests/python5.test | 3 +- tests/python6.test | 3 +- tests/python7.test | 3 +- tests/python8.test | 3 +- tests/python9.test | 3 +- tests/sanity.test | 6 +-- tests/subobj.test | 3 +- tests/subobj9.test | 4 +- tests/unused.test | 4 +- tests/vars3.test | 3 +- tests/version8.test | 3 +- 56 files changed, 312 insertions(+), 168 deletions(-) diff --git a/ChangeLog b/ChangeLog index 152e0c6..0df5ec3 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,91 @@ +2009-09-09 Stefano Lattarini + + Fix testsuite: avoid Zsh-related problems and glitches, especially + those due to a Zsh "bug" w.r.t. `set -x'. + * tests/README: Describe a bug (or at least a weird behaviour) in + the way the 4.x versions of Zsh deal with the `-x' a.k.a. `xtrace' + switch. This bug can unexpectedly break apparently flawless test + scripts. Describe also a workaround for it, involving the use of + the new shell function `run_CMD()', defined in tests/defs.in (see + below). + * tests/defs.in: New subroutine `run_CMD()'. Implementation of + subroutine `AUTOMAKE_run()' rewritten to use `run_CMD()'. Unset + variable `TEST_LOG_COMPILER', that might cause spurious failures + by leaking in the environment of the make processes executed by the + test scripts. Updated the code which ensures Bourne-compatibility + in Zsh, by adding a call to `setopt NO_GLOB_SUBST' (as done by + autoconf 2.64). + * tests/acloca14.test: + * tests/acloca17.test: + * tests/acloca18.test: + * tests/aclocal.test: + * tests/aclocal3.test: + * tests/aclocal8.test: + * tests/acsilent.test: + * tests/alpha.test: + * tests/ammissing.test: + * tests/ansi3.test: + * tests/ansi3b.test: + * tests/check8.test: + * tests/cond39.test: + * tests/defs.in: + * tests/depcomp2.test: + * tests/depcomp4.test: + * tests/depend5.test: + * tests/depend6.test: + * tests/discover.test: + * tests/distcom3.test: + * tests/fn99.test: + * tests/fn99subdir.test: + * tests/gnits2.test: + * tests/gnits3.test: + * tests/help.test: + * tests/init.test: + * tests/instsh.test: + * tests/ldadd.test: + * tests/libtool7.test: + * tests/libtool9.test: + * tests/lisp3.test: + * tests/ltorder.test: + * tests/man4.test: + * tests/mdate.test: + * tests/missing3.test: + * tests/missing4.test: + * tests/missing6.test: + * tests/obsolete.test: + * tests/output-order.test: + * tests/pr220.test: + * tests/pr300-ltlib.test: + * tests/python11.test: + * tests/python4.test: + * tests/python5.test: + * tests/python6.test: + * tests/python7.test: + * tests/python8.test: + * tests/python9.test: + * tests/sanity.test: + * tests/subobj.test: + * tests/subobj9.test: + * tests/unused.test: + * tests/vars3.test: + * tests/version8.test: + use new subroutine `run_CMD()' instead of hand-crafted stdout and/or + stderr redirections; this, togheter with the changes in file + `tests/defs.in', fixes misbehaviour w.r.t. zsh for (at least) the + following tests: + - aclocal8.test + - acsilent.test + - depcomp2.test + - getopt.test + - location.test + - missing3.test + - output-order.test + - parallel-tests3.test + - parallel-tests4.test + - unused.test + - warnopts.test + - check9-p.test + 2009-09-06 Ralf Wildenhues Avoid sleeping for one second most of the time in sanity check. diff --git a/tests/README b/tests/README index b264881..a99a305 100644 --- a/tests/README +++ b/tests/README @@ -157,6 +157,29 @@ Do not reason, but at least it makes sure the original error is still here.) + If you must run a program and later analize its stderr, do *not* run it + by doing e.g.: + [WRONG] PROG 2>foo.txt + Instead, use the run_CMD() subroutine (defined in tests/defs), as in: + [GOOD] run_CMD PROG; mv stderr foo.txt + You can't use the apparently simpler form `PROG 2>stderr.txt' because of + a nasty bug of the zsh shell (at least of some 4.x versions). + In details: if you run command when `set -x' is active, and redirect its + standard error to a file, then zsh writes also the *trace* for the + command into that file, instead that into the original standard error. + For example: + $ zsh -c -x 'true 2>file'; echo "SHOULD BE EMPTY: `cat file`" + SHOULD BE EMPTY: +zsh:1> true + while the correct behaviour is e.g. + $ ksh -c -x 'true 2>file'; echo "SHOULD BE EMPTY: `cat file`" + + true + + 2> file + SHOULD BE EMPTY: + or: + $ bash -c -x 'true 2>file'; echo "SHOULD BE EMPTY: `cat file`" + + true + SHOULD BE EMPTY: + Do not override Makefile variables using make arguments, as in $MAKE ANSI2KNR=./ansi2knr U=_ all this is not portable for recursive targets (targets that diff --git a/tests/acloca14.test b/tests/acloca14.test index 8ddbb1a..6df7c9b 100755 --- a/tests/acloca14.test +++ b/tests/acloca14.test @@ -108,5 +108,5 @@ $MAKE testdist2 # Make sure aclocal diagnose missing included files with correct `file:line:'. rm -f b.m4 -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e 1 $ACLOCAL grep 'a.m4:1:.*b.m4.*does not exist' stderr diff --git a/tests/acloca17.test b/tests/acloca17.test index 884f24b..a8326f9 100755 --- a/tests/acloca17.test +++ b/tests/acloca17.test @@ -33,7 +33,5 @@ EOF # FIXME: We want autom4te's 'undefined required macro' warning to be fatal, # but have no means to say so to aclocal. We use WARNINGS=error instead. - -WARNINGS=error $ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e 1 env WARNINGS=error $ACLOCAL -I m4 grep 'configure.in:4:.*UNDEFINED_MACRO' stderr diff --git a/tests/acloca18.test b/tests/acloca18.test index 22feb23..e749b73 100755 --- a/tests/acloca18.test +++ b/tests/acloca18.test @@ -86,7 +86,7 @@ grep macro23 foo ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1 -I 2' rm -f foo -$ACLOCAL --install 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e 1 $ACLOCAL --install grep AM_MACRO2 stderr ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1' @@ -105,9 +105,7 @@ AC_DEFUN([AM_MACRO1], [echo macro1d >> foo]) AC_DEFUN([AM_MACRO2], [echo macro2d >> foo]) EOF rm -f foo -$ACLOCAL --diff=diff >output 2>stderr -cat stderr -cat output -grep '#serial 456' output +run_CMD -e 0 $ACLOCAL --diff=diff +grep '#serial 456' stdout test ! -f 4/m1.m4 grep 'installing.*4/m1.m4' stderr diff --git a/tests/aclocal.test b/tests/aclocal.test index 6b87a9d..fbb6134 100755 --- a/tests/aclocal.test +++ b/tests/aclocal.test @@ -23,15 +23,15 @@ set -e $ACLOCAL --output=fred test -f fred -$ACLOCAL --output 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e 1 $ACLOCAL --output grep 'option.*--output.*an argument' stderr grep help stderr -$ACLOCAL --unknown-option 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e 1 $ACLOCAL --unknown-option grep 'unrecognized.*--unknown-option' stderr grep help stderr -$ACLOCAL --ver 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e 1 $ACLOCAL --ver grep 'unrecognized.*--ver' stderr grep help stderr diff --git a/tests/aclocal3.test b/tests/aclocal3.test index a550e50..2474304 100755 --- a/tests/aclocal3.test +++ b/tests/aclocal3.test @@ -27,6 +27,5 @@ AC_DEFUN([GNOME_X_CHECKS], [ ]) END -$ACLOCAL -I macros 2>stderr -cat stderr +run_CMD -e 0 $ACLOCAL -I macros grep 'macros/gnome.m4:2:.*AM_PATH_GTK.*not found' stderr diff --git a/tests/aclocal8.test b/tests/aclocal8.test index ac10aff..935760a 100755 --- a/tests/aclocal8.test +++ b/tests/aclocal8.test @@ -34,9 +34,8 @@ EOF echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4 echo 'AC_DEFUN([MACRO2], [AC_REQUIRE([AM_UNUSED_MACRO])])' >m4/macro2.m4 -$ACLOCAL -I m4 >output 2>&1 -cat output -test 0 = `wc -l output 2>&1 || { cat output; Exit 1; } -test -z "`cat output`" +run_CMD -e 0 -m $ACLOCAL +test -s stdall && Exit 1 +: diff --git a/tests/alpha.test b/tests/alpha.test index 1ee8ed6..f1829ec 100755 --- a/tests/alpha.test +++ b/tests/alpha.test @@ -62,8 +62,7 @@ $AUTOMAKE ./configure # make distdir should fail because NEWS does not mention 1.0a -$MAKE check 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAIL $MAKE check grep 'NEWS not updated' stderr test ! -f works diff --git a/tests/ammissing.test b/tests/ammissing.test index 61d360a..ad7b8cd 100755 --- a/tests/ammissing.test +++ b/tests/ammissing.test @@ -20,6 +20,5 @@ echo AM_ZARDOZ >> configure.in -$ACLOCAL 2>stderr -cat stderr +run_CMD -e 0 $ACLOCAL grep 'configure.in:.*AM_ZARDOZ.*not found' stderr diff --git a/tests/ansi3.test b/tests/ansi3.test index 2bbcaf7..7d7fec0 100755 --- a/tests/ansi3.test +++ b/tests/ansi3.test @@ -53,8 +53,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_CMD -e 0 $AUTOMAKE -a grep 'Makefile.am:1:.*ansi2knr\.c' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ansi3b.test b/tests/ansi3b.test index 8f254ad..cba7a73 100755 --- a/tests/ansi3b.test +++ b/tests/ansi3b.test @@ -52,8 +52,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_CMD -e 0 $AUTOMAKE -a grep 'configure.in:2:.*ansi2knr\.1' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/check8.test b/tests/check8.test index 7936582..d1d07e5 100755 --- a/tests/check8.test +++ b/tests/check8.test @@ -67,10 +67,7 @@ $AUTOMAKE -a unset TESTS || : ./configure -AM_COLOR_TESTS=always $MAKE -e check >stdout 2>stderr && - { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_CMD -e FAIL -- env AM_COLOR_TESTS=always $MAKE -e check grep 'XPASS.* foo$' stdout grep '^[^X]*PASS.* sub/foo$' stdout grep '^[^X]*PASS.* bar' stdout @@ -87,8 +84,7 @@ $MAKE distclean mkdir build cd build ../configure -$MAKE check >stdout && { cat stdout; Exit 1; } -cat stdout +run_CMD -e FAIL $MAKE check # Note: we are not grepping for the space here, due to the Solaris make VPATH # rewriting (if we fix that, we can still write a separate test for it). grep 'XPASS.*foo$' stdout diff --git a/tests/cond39.test b/tests/cond39.test index 79eaa70..a06969b 100755 --- a/tests/cond39.test +++ b/tests/cond39.test @@ -82,8 +82,7 @@ $AUTOCONF $AUTOMAKE --add-missing ./configure COND=true -$MAKE 2>stderr -cat stderr +run_CMD -e 0 $MAKE grep 'overriding commands' stderr && Exit 1 $MAKE sure-exist ./prog1 && Exit 1 @@ -97,8 +96,7 @@ $MAKE sure-exist $MAKE distclean ./configure COND=false -$MAKE 2>stderr -cat stderr +run_CMD -e 0 $MAKE grep 'overriding commands' stderr && Exit 1 ./prog1 ./sub/prog2 diff --git a/tests/defs.in b/tests/defs.in index 61182b6..67c95d4 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -20,19 +20,19 @@ # Defines for Automake testing environment. # Tom Tromey -# Be Bourne compatible. -# (Snippet copied from configure's initialization in Autoconf 2.59c.) -if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then +# Be more Bourne compatible +# (Snippet copied from configure's initialization in Autoconf 2.64) +DUALCASE=1; export DUALCASE # for MKS sh +if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: - # Zsh 3.x and 4.x performs word splitting on ${1+"$@"}, which + # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which # is contrary to our usage. Disable this feature. alias -g '${1+"$@"}'='"$@"' + setopt NO_GLOB_SUBST else - case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac + case `(set -o) 2>/dev/null` in *posix*) set -o posix ;; esac fi -BIN_SH=xpg4; export BIN_SH # for Tru64 -DUALCASE=1; export DUALCASE # for MKS sh # Ensure we are running from the right directory. test -f ./defs || { @@ -314,6 +314,7 @@ unset oldincludedir pdfdir psdir sbindir sharedstatedir sysconfdir # backward-compatible behavior. unset DISABLE_HARD_ERRORS unset TESTS +unset TEST_LOG_COMPILER unset TEST_LOGS unset RECHECK_LOGS unset VERBOSE @@ -396,26 +397,100 @@ is_newest () test -z "$is_newest_files" } +# run_CMD [-e STATUS] [-i FILE] [-m] [--] COMMAND [ARGUMENTS..] +# ------------------------------------------------------------- +# Run the given COMMAND (can be an external commnds, a shell function or +# shell builtin) with ARGUMENTS (and with standard input taken from FILE, +# if option `-i' is given), and fail if COMMAND does not exit with STATUS. +# If status is "VOID" or "IGNORE", any exit value of the command is +# acceptable. If STATUS is "FAIL", then any exit value of the command +# *but 0* is acceptable. Default STATUS is "IGNORE". +# Also, save standard output and standard error from COMMAND, by default +# respectively in files `stdout' and `stderr' (in the current directory), +# or togheter in the file `stdall' (in the current directory) if the `-m' +# option is given. All the redirections are done without triggering the +# zsh4 bug related to `-x' shell switch (described in details in the +# tests/README file). +run_CMD () +{ + # NOTE: all internal variables used here starts with the `_run' + # prefix, to minimize possibility of name clashes with global + # variables defined in user code. + : 'entering run_CMD(): become quiet' + set +x # xtrace verbosity stops here + _run_stdin=- + _run_expected_exitcode=0 + _run_mix_stdout_and_stderr=no + while test $# -gt 0; do + case "$1" in + -e) _run_expected_exitcode=$2; shift;; + -i) _run_stdin=$2; shift;; + -m) _run_mix_stdout_and_stderr=yes;; + --) shift; break;; + -?) echo "run_CMD(): invalid switch '$1'" >&2; Exit 99;; + *) break;; + esac + shift + done + case $# in + 0) echo "run_CMD(): missing COMMAND argument" >&2; Exit 99;; + *) _run_cmd=$1; shift;; + esac + _run_exitcode=0 + if test x"${_run_mix_stdout_and_stderr}" = x"yes"; then + _run_evald_cmd='"${_run_cmd}" ${1+"$@"} >stdall 2>&1' + else + _run_evald_cmd='"${_run_cmd}" ${1+"$@"} >stdout 2>stderr' + fi + if test x"${_run_stdin}" != x"-"; then + _run_evald_cmd="${_run_evald_cmd}"' <"${_run_stdin}"' + fi + _run_evald_cmd="${_run_evald_cmd} || _run_exitcode=\$?" + eval "${_run_evald_cmd}" + if test x"${_run_mix_stdout_and_stderr}" = x"yes"; then + echo "=== stdout and stderr" + cat stdall + echo "===" + else + echo "=== stderr" >&2 + cat stderr >&2 + echo "===" >&2 + echo "=== stdout" + cat stdout + echo "===" + fi + case ${_run_expected_exitcode} in + VOID|void|IGNORE|ignore|IGNORED|ignored|${_run_exitcode}) + _run_rc=0 + ;; + FAIL|fail|FAILURE|failure) + test ${_run_exitcode} -gt 0 && _run_rc=0 || _run_rc=1 + ;; + *) + _run_rc=1 + ;; + esac + set -x # xtrace verbosity restart here + : "exit status ${_run_exitcode} (expecting ${_run_expected_exitcode})" + : "leaving run_CMD()" + return ${_run_rc} +} # AUTOMAKE_run status [options...] # -------------------------------- -# Run Automake with OPTIONS, and fail if automake +# Run Automake with OPTIONS, and make the testcase FAIL if automake # does not exit with STATUS. AUTOMAKE_run () { - expected_exitcode=$1 + _am_run_expected_exitcode=$1 shift - exitcode=0 - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? - cat stderr >&2 - cat stdout - test $exitcode = $expected_exitcode || Exit 1 + run_CMD -e ${_am_run_expected_exitcode} -- $AUTOMAKE ${1+"$@"} || Exit 1 } # AUTOMAKE_fails [options...] # --------------------------- -# Run Automake with OPTIONS, and fail if automake -# does not exit with STATUS. +# Run Automake with OPTIONS, and make the testcase FAIL if automake +# does not exit with status 1. AUTOMAKE_fails () { AUTOMAKE_run 1 ${1+"$@"} diff --git a/tests/depcomp2.test b/tests/depcomp2.test index 4653c90..0969c0f 100755 --- a/tests/depcomp2.test +++ b/tests/depcomp2.test @@ -20,6 +20,8 @@ required=gcc . ./defs || Exit 1 +set -e + cat > configure.in << 'END' AC_INIT(subdir/foo.c) AM_INIT_AUTOMAKE(nonesuch, nonesuch) @@ -48,5 +50,6 @@ export CFLAGS $ACLOCAL || Exit 1 $AUTOMAKE --add-missing || Exit 1 $AUTOCONF || Exit 1 -./configure CC='gcc' 2>error.log -test -z "`cat error.log`" +run_CMD -e 0 ./configure CC='gcc' +test -s stderr && Exit 1 +: diff --git a/tests/depcomp4.test b/tests/depcomp4.test index 7d22894..29f63a0 100755 --- a/tests/depcomp4.test +++ b/tests/depcomp4.test @@ -51,7 +51,6 @@ $AUTOMAKE -a grep am_cv_CC_dependencies_compiler_type configure ./configure am_cv_CC_dependencies_compiler_type=makedepend -$MAKE >output 2>&1 -cat output -grep 'unknown directive' output && Exit 1 +run_CMD -e 0 -m $MAKE +grep 'unknown directive' stdall && Exit 1 $MAKE grepdeps diff --git a/tests/depend5.test b/tests/depend5.test index 0d9be3b..5f9379a 100755 --- a/tests/depend5.test +++ b/tests/depend5.test @@ -65,10 +65,7 @@ for arg in Makefile \ "--file=$name" do rm -rf .deps _deps - ./config.status "$arg" depfiles >stdout 2>stderr || - { cat stdout; cat stderr >&2; Exit 1; } - cat stdout - cat stderr >&2 + run_CMD -e 0 ./config.status "$arg" depfiles grep '[Nn]o such file' stderr && Exit 1 if test -n "$depdir"; then diff --git a/tests/depend6.test b/tests/depend6.test index de2adf9..ce3a6df 100755 --- a/tests/depend6.test +++ b/tests/depend6.test @@ -39,9 +39,6 @@ END $ACLOCAL $AUTOMAKE $AUTOCONF -./configure --disable-dependency-tracking 2>stderr || { - stat=$?; cat stderr >&2; Exit $stat; -} -cat stderr >&2 -grep shift stderr && Exit 1 +run_CMD -e 0 ./configure --disable-dependency-tracking +grep 'shift' stderr && Exit 1 : diff --git a/tests/discover.test b/tests/discover.test index 36293f7..6e8bf37 100755 --- a/tests/discover.test +++ b/tests/discover.test @@ -34,5 +34,6 @@ END : > fsusage.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | $FGREP fsusage.c || Exit 1 +run_CMD -e IGNORE -m $AUTOMAKE +$FGREP fsusage.c stdall || Exit 1 Exit 0 diff --git a/tests/distcom3.test b/tests/distcom3.test index 04bf365..314c6eb 100755 --- a/tests/distcom3.test +++ b/tests/distcom3.test @@ -34,9 +34,8 @@ END $ACLOCAL # Should not warn about missing README, since it is a target. -$AUTOMAKE --add-missing --gnu >stdout 2>&1 -cat stdout -grep README stdout && Exit 1 +run_CMD -e 0 -m $AUTOMAKE --add-missing --gnu +grep README stdall && Exit 1 sed -n -e '/^DIST_COMMON =.*\\$/ { :loop diff --git a/tests/fn99.test b/tests/fn99.test index 982a989..6b45e15 100755 --- a/tests/fn99.test +++ b/tests/fn99.test @@ -47,8 +47,7 @@ do touch x done) || Exit 77 -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAIL $MAKE dist grep 'filenames are too long' stderr test 2 = `grep 12345678 stderr | wc -l` : diff --git a/tests/fn99subdir.test b/tests/fn99subdir.test index 465f2af..f9db1f5 100755 --- a/tests/fn99subdir.test +++ b/tests/fn99subdir.test @@ -73,8 +73,7 @@ do touch x done) -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAIL $MAKE dist grep 'filenames are too long' stderr test 1 = `grep 12345678 stderr | wc -l` : diff --git a/tests/gnits2.test b/tests/gnits2.test index 4c5d54f..df9124c 100755 --- a/tests/gnits2.test +++ b/tests/gnits2.test @@ -105,6 +105,6 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE all $MAKE test-install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_CMD -e IGNORE $MAKE -k installcheck $MAKE grep-stderr diff --git a/tests/gnits3.test b/tests/gnits3.test index 7a08fb8..3a164bd 100755 --- a/tests/gnits3.test +++ b/tests/gnits3.test @@ -83,8 +83,8 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE $MAKE install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_CMD -e IGNORE $MAKE -k installcheck $MAKE grep-stderr # Make sure there is no more error when all targets are exempted. diff --git a/tests/help.test b/tests/help.test index d4d7110..0eefc52 100755 --- a/tests/help.test +++ b/tests/help.test @@ -32,7 +32,7 @@ $AUTOMAKE --version $AUTOMAKE --help # aclocal and automake cannot work without configure.ac or configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e 1 $ACLOCAL grep configure.ac stderr grep configure.in stderr AUTOMAKE_fails diff --git a/tests/init.test b/tests/init.test index 38ec681..f2a472c 100755 --- a/tests/init.test +++ b/tests/init.test @@ -27,7 +27,11 @@ AM_INIT_AUTOMAKE END # The error message should mension AC_INIT, not AC_PACKAGE_VERSION. -($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAILURE $ACLOCAL || aclocal_success=yes +mv stderr aclocal-stderr +run_CMD -e FAILURE $AUTOCONF || autoconf_success=yes +mv stderr autoconf-stderr +test x"$aclocal_sucess#$autoconf_success" = x"yes#yes" && Exit 1 +cat aclocal-stderr autoconf-stderr > stderr grep AC_PACKAGE_VERSION stderr && Exit 1 grep AC_INIT stderr diff --git a/tests/instsh.test b/tests/instsh.test index 61f5f9e..0b06c46 100755 --- a/tests/instsh.test +++ b/tests/instsh.test @@ -32,10 +32,9 @@ mv Makefile.am configure.in frob/ cd frob $ACLOCAL -$AUTOMAKE --add-missing > output 2>&1 +run_CMD -e 0 -m $AUTOMAKE --add-missing # Only one `/' should appear in the output. -cat output -grep '/.*/' output && Exit 1 +grep '/.*/' stdall && Exit 1 test -f install-sh diff --git a/tests/ldadd.test b/tests/ldadd.test index 005f9d8..87200e5 100755 --- a/tests/ldadd.test +++ b/tests/ldadd.test @@ -39,5 +39,6 @@ END : > q.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | grep LDADD || Exit 1 +run_CMD -e 1 -m $AUTOMAKE +grep LDADD stdall || Exit 1 Exit 0 diff --git a/tests/libtool7.test b/tests/libtool7.test index a2f912f..dfaaa36 100755 --- a/tests/libtool7.test +++ b/tests/libtool7.test @@ -82,19 +82,16 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure "--prefix=`pwd`/_inst" -env LIBTOOLFLAGS=--silent $MAKE print >output 2>&1 -cat output -grep '1BEG: libmod1.la mod2.la :END1' output -grep '2BEG: mod2.la :END2' output -grep '3BEG: .*silent.*silent.* :END3' output +run_CMD -e 0 -m env LIBTOOLFLAGS=--silent $MAKE print +grep '1BEG: libmod1.la mod2.la :END1' stdall +grep '2BEG: mod2.la :END2' stdall +grep '3BEG: .*silent.*silent.* :END3' stdall test 2 -le `grep mod2_la_LIBTOOLFLAGS Makefile | wc -l` $MAKE -env LIBTOOLFLAGS=--silent $MAKE install >output 2>&1 -cat output -grep 'silent.*silent.*prg' output -grep 'silent.*silent.*libmod1' output +run_CMD -e 0 -m env LIBTOOLFLAGS=--silent $MAKE install +grep 'silent.*silent.*prg' stdall +grep 'silent.*silent.*libmod1' stdall -env LIBTOOLFLAGS=--silent $MAKE uninstall >output 2>&1 -cat output -grep 'silent.*silent.*libmod1' output +run_CMD -e 0 -m env LIBTOOLFLAGS=--silent $MAKE uninstall +grep 'silent.*silent.*libmod1' stdall diff --git a/tests/libtool9.test b/tests/libtool9.test index c0d9688..b46315d 100755 --- a/tests/libtool9.test +++ b/tests/libtool9.test @@ -88,10 +88,11 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure -env LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ +run_CMD -e 0 -m env \ + LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ CFLAGS=cflags AM_CFLAGS=am_cflags prg2_CFLAGS=prg2_cflags \ - $MAKE -e print >output 2>&1 -cat output + $MAKE -e print +mv stdall output grep '1BEG: libmod1.la mod2.la :END1' output grep '2BEG: mod2.la :END2' output grep '3BEG:.* am_cflags cflags .*lm1_la_ldflags ldflags.* :END3' output diff --git a/tests/lisp3.test b/tests/lisp3.test index 809d491..4a94e0e 100755 --- a/tests/lisp3.test +++ b/tests/lisp3.test @@ -61,9 +61,8 @@ test -f am-one.elc unique=0a3346e2af8a689b85002b53df09142a $sleep echo "(message \"$unique\")(provide 'am-three)" > am-three.el -$MAKE >output 2>&1 -cat output -grep $unique output +run_CMD -e 0 -m $MAKE +grep $unique stdall # It should also work for VPATH-builds. $MAKE distcheck diff --git a/tests/ltorder.test b/tests/ltorder.test index 0f69211..52fcbbe 100755 --- a/tests/ltorder.test +++ b/tests/ltorder.test @@ -53,8 +53,7 @@ $AUTOMAKE --add-missing ./configure "--prefix=`pwd`/inst" $MAKE -$MAKE install 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e 0 $MAKE install grep 'has not been installed' stderr && Exit 1 $MAKE uninstall diff --git a/tests/man4.test b/tests/man4.test index aa86a9e..a6eaa66 100755 --- a/tests/man4.test +++ b/tests/man4.test @@ -88,11 +88,9 @@ mkdir build cd build ../configure $MAKE -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAIL $MAKE dist grep 'install help2man' stderr -$MAKE distcheck 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAIL $MAKE distcheck grep 'install help2man' stderr rm -f ../help2man diff --git a/tests/mdate.test b/tests/mdate.test index 704b393..0751f89 100755 --- a/tests/mdate.test +++ b/tests/mdate.test @@ -31,4 +31,5 @@ END : > texinfo.tex $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 > /dev/null | grep 'required file.*mdate-sh' +run_CMD -e IGNORED $AUTOMAKE +grep 'required file.*mdate-sh' stderr diff --git a/tests/missing3.test b/tests/missing3.test index 07ef779..d0b3c29 100755 --- a/tests/missing3.test +++ b/tests/missing3.test @@ -22,24 +22,24 @@ set -e # b7cb8259 assumed not to exist. -./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e FAIL ./missing b7cb8259 --version grep . stderr && Exit 1 -./missing b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e FAIL ./missing b7cb8259 --grep grep WARNING stderr -./missing --run b7cb8259 --version && Exit 1 -./missing --run b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e FAIL ./missing --run b7cb8259 --version +run_CMD -e FAIL ./missing --run b7cb8259 --grep grep WARNING stderr # missing itself it known to exist :) -./missing ./missing --version 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e FAIL ./missing ./missing --version grep . stderr && Exit 1 -./missing ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e FAIL ./missing ./missing --grep grep WARNING stderr -./missing --run ./missing --version 2>stderr +run_CMD -e 0 ./missing --run ./missing --version grep . stderr && Exit 1 -./missing --run ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_CMD -e FAIL ./missing --run ./missing --grep grep WARNING stderr && Exit 1 grep Unknown stderr diff --git a/tests/missing4.test b/tests/missing4.test index 1015828..2ac66da 100755 --- a/tests/missing4.test +++ b/tests/missing4.test @@ -36,13 +36,11 @@ cmp aclocal.m4 aclocal.tmp && Exit 1 mv aclocal.tmp aclocal.m4 -$MAKE 2>stderr -cat stderr +run_CMD -e 0 $MAKE grep 'You have another version of autoconf' stderr grep -c 'aclocal.m4:.*this file was generated for' stderr -$MAKE 2>stderr -cat stderr +run_CMD -e 0 $MAKE grep 'You have another version of autoconf' stderr && Exit 1 grep 'aclocal.m4:.*this file was generated for' stderr && Exit 1 diff --git a/tests/missing6.test b/tests/missing6.test index b63a0d1..096d904 100755 --- a/tests/missing6.test +++ b/tests/missing6.test @@ -31,7 +31,7 @@ rm -f configure.in touch Makefile.am $ACLOCAL -$AUTOCONF 2>stderr +run_CMD -e 0 $AUTOCONF grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for' stderr $AUTOMAKE @@ -43,8 +43,7 @@ cmp configure.ac configure.tmp && Exit 1 mv configure.tmp configure.ac -$MAKE 2>stderr -cat stderr +run_CMD -e 0 $MAKE grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for autoconf 9999a' stderr diff --git a/tests/obsolete.test b/tests/obsolete.test index e6601c9..46222d2 100755 --- a/tests/obsolete.test +++ b/tests/obsolete.test @@ -27,7 +27,7 @@ END $PERL -ne '/AU_DEFUN\(\[(\w+)\]/ && print "$1\n"' \ "$testsrcdir/../m4/obsolete.m4" >> obs cat obs >> configure.in -$PERL -ne 'chomp; print "grep $_ stderr || Exit 1\n"; ' obs > obs.1 +$PERL -ne 'chomp; print "grep $_ stdall || Exit 1\n"; ' obs > obs.1 $PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2 # Sanity check. Make sure we have added something to configure.in. @@ -36,7 +36,7 @@ test `cat configure.in | wc -l` -gt 1 || Exit 1 $ACLOCAL || Exit 1 # Expect Autoconf to complain about each of the macros in obs. -$AUTOCONF -Wobsolete >stderr 2>&1 +run_CMD -e 0 -m $AUTOCONF -Wobsolete . ./obs.1 # Make sure Autoupdate remove each of these macros. $AUTOUPDATE || Exit 1 diff --git a/tests/output-order.test b/tests/output-order.test index 856baa2..cde0c64 100755 --- a/tests/output-order.test +++ b/tests/output-order.test @@ -43,13 +43,12 @@ END $ACLOCAL $AUTOCONF rm -f missing install-sh -$AUTOMAKE --add-missing --copy 2>stderr -cat stderr >&2 +run_CMD -e 0 $AUTOMAKE --add-missing --copy +mv stderr stder0 for i in 1 2 3 4 5 6; do rm -f missing install-sh INSTALL COPYING + run_CMD -e IGNORE $AUTOMAKE --add-missing --copy # The grep prevents a Heisenbug with the HP-UX shell and VERBOSE=yes. - $AUTOMAKE --add-missing --copy 2>&1 >/dev/null | - grep -v /dev/null | - diff - stderr + grep -v /dev/null stder0 | diff - stderr done diff --git a/tests/pr220.test b/tests/pr220.test index 25e1f0e..89272b6 100755 --- a/tests/pr220.test +++ b/tests/pr220.test @@ -55,6 +55,5 @@ $AUTOMAKE -a cd build # configure should fail since we've done something invalid. -../configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAIL ../configure grep NEVER_TRUE stderr diff --git a/tests/pr300-ltlib.test b/tests/pr300-ltlib.test index 01814a9..3e606c4 100755 --- a/tests/pr300-ltlib.test +++ b/tests/pr300-ltlib.test @@ -48,8 +48,7 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --copy --add-missing ./configure --prefix "`pwd`/inst" -$MAKE V=1 >stdout -cat stdout +run_CMD -e 0 $MAKE V=1 grep 'liba.la .*-rpath .*lib' stdout grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1 @@ -58,8 +57,7 @@ grep 'libb.la .*-rpath .*lib/subdir' stdout test -f subdir/liba.la test -f subdir/libb.la -$MAKE install 2>stderr -cat stderr >&2 +run_CMD -e 0 $MAKE install grep 'remember.*--finish' stderr && Exit 1 test -f inst/lib/liba.la diff --git a/tests/python11.test b/tests/python11.test index 0f15f37..af880bb 100755 --- a/tests/python11.test +++ b/tests/python11.test @@ -35,9 +35,7 @@ EOF $ACLOCAL $AUTOCONF -./configure >stdout 2>stderr && { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_CMD -e FAILURE ./configure grep 'checking for IShouldNotExist1' stdout grep 'checking for IShouldNotExist2' stdout grep 'no suitable Python interpreter found' stderr diff --git a/tests/python4.test b/tests/python4.test index 11bb3ed..efd23ce 100755 --- a/tests/python4.test +++ b/tests/python4.test @@ -33,6 +33,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAILURE ./configure PYTHON=: grep 'no suitable Python interpreter found' stderr diff --git a/tests/python5.test b/tests/python5.test index 86f5863..a508ac9 100755 --- a/tests/python5.test +++ b/tests/python5.test @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_CMD -e FAILURE ./configure grep 'no suitable Python interpreter found' stderr diff --git a/tests/python6.test b/tests/python6.test index eb4fb4d..c08833d 100755 --- a/tests/python6.test +++ b/tests/python6.test @@ -34,6 +34,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr -cat stderr +run_CMD -e 0 ./configure PYTHON=: grep 'GREP ME:' stderr diff --git a/tests/python7.test b/tests/python7.test index 62c1a1b..4999e02 100755 --- a/tests/python7.test +++ b/tests/python7.test @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_CMD -e 0 ./configure grep 'GREP ME:' stderr diff --git a/tests/python8.test b/tests/python8.test index a50af22..d8f0ebd 100755 --- a/tests/python8.test +++ b/tests/python8.test @@ -32,6 +32,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_CMD -e 0 ./configure grep 'GREP ME' stderr diff --git a/tests/python9.test b/tests/python9.test index 9b23deb..70e0143 100755 --- a/tests/python9.test +++ b/tests/python9.test @@ -33,6 +33,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_CMD -e 0 ./configure grep 'GREP ME' stderr diff --git a/tests/sanity.test b/tests/sanity.test index 1c96c78..a16a547 100755 --- a/tests/sanity.test +++ b/tests/sanity.test @@ -36,14 +36,12 @@ cp ../install-sh ../missing . $ACLOCAL $AUTOCONF $AUTOMAKE -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_CMD -e FAILURE ./configure grep 'unsafe absolute working directory' stderr cd .. mkdir build cd build -../unsafe$/configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_CMD -e FAILURE ../unsafe$/configure grep 'unsafe srcdir' stderr : diff --git a/tests/subobj.test b/tests/subobj.test index df297f4..5a67b54 100755 --- a/tests/subobj.test +++ b/tests/subobj.test @@ -32,8 +32,7 @@ wish_SOURCES = generic/a.c generic/b.c END $ACLOCAL -$AUTOMAKE --add-missing 2>stderr -cat stderr +run_CMD -e 0 $AUTOMAKE --add-missing # Make sure compile is installed, and that Automake says so. grep 'install.*compile' stderr test -f compile diff --git a/tests/subobj9.test b/tests/subobj9.test index 99fe6e8..bf6ec37 100755 --- a/tests/subobj9.test +++ b/tests/subobj9.test @@ -74,8 +74,8 @@ if test -n "`./libtool --help | grep tag=TAG`"; then fi $MAKE -$MAKE distcheck 2>&1 | tee out +run_CMD -e 0 -m $MAKE distcheck # GNU Make used to complain that the Makefile contained two rules # for `src/.dirstamp' and `.//src/.dirstamp'. -grep 'overriding commands' out && Exit 1 +grep 'overriding commands' stdall && Exit 1 : diff --git a/tests/unused.test b/tests/unused.test index a5b4da6..d04762a 100755 --- a/tests/unused.test +++ b/tests/unused.test @@ -29,5 +29,5 @@ AC_DEFUN([MACRO_1_2], echo 12) AC_DEFUN([MACRO_1_2_3], echo 123) END -$ACLOCAL 2> stderr || { cat stderr >&2; Exit 1; } -test -z "`cat stderr`" +run_CMD -e 0 $ACLOCAL +test ! -s stderr diff --git a/tests/vars3.test b/tests/vars3.test index 3d10a90..d79cb08 100755 --- a/tests/vars3.test +++ b/tests/vars3.test @@ -38,8 +38,7 @@ EOF $ACLOCAL # Make sure this warning is print in the `portability' category. -$AUTOMAKE --warnings=no-error,none,portability 2>stderr -cat stderr +run_CMD -e 0 $AUTOMAKE --warnings=no-error,none,portability # Lines number are printed in error message. # Use them to make sure errors are diagnosed against the right lines. diff --git a/tests/version8.test b/tests/version8.test index 91b11a0..fef8e2f 100755 --- a/tests/version8.test +++ b/tests/version8.test @@ -20,6 +20,5 @@ set -e echo 'AM_AUTOMAKE_VERSION([1.9])' >>configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 0; } -cat stderr >&2 +run_CMD -e 1 $ACLOCAL $FGREP 'AM_INIT_AUTOMAKE([1.9])' stderr -- 1.6.3.3