[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-983-gfda3de5 |
|
Date: |
Sun, 07 Aug 2011 23:30:18 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".
http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=fda3de574a6c8398420ad6ae1f456aad368a7193
The branch, test-protocols has been updated
via fda3de574a6c8398420ad6ae1f456aad368a7193 (commit)
via 152fcd209b21d8a588e787d60abd59d74bbeb69b (commit)
via 0ef3ef52079a967a4ef15968fb2054f1b215e7fb (commit)
via 1072f686df5bb1035620af4a22007feb290d1f12 (commit)
via b7e96983128caec94b256760ae9694c262775988 (commit)
via 42851b794a9bed9fdbb7240b551ebd6bd55f77e7 (commit)
via 30bef0f3dc2e56b1205e3a1d69f5321a9c7a99e6 (commit)
via 4740f56e6fe3360fc21e5de68600a777fd046e99 (commit)
via e0da0d0740c8174270dae2eb2d5fc1f3e8c6c041 (commit)
from c7fa87264291f611fdc63bd4a49913f98ec5b944 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit fda3de574a6c8398420ad6ae1f456aad368a7193
Author: Stefano Lattarini <address@hidden>
Date: Mon Aug 8 01:23:42 2011 +0200
test harness: work around a VPATH rewrite issue
* lib/am/check.am (am__TEST_BASES): Removed, it's role taken
over by ...
(am__set_TESTS_bases): ... these new variable.
($(TEST_SUITE_LOG): Use it, to avoid VPATH rewrite issues.
* automake.in (handle_tests): Update the code for the cleanup
of the `.trs' file to use `$(TEST_LOGS)' instead of
`$(am__TEST_BASES)'.
* tests/test-harness-vpath-rewrite.test: New test, better
exposing the problem fixed by this change
* tests/Makefile.am (TESTS): Add it.
* tests/test-trs-basic.test: Update and extend.
commit 152fcd209b21d8a588e787d60abd59d74bbeb69b
Author: Stefano Lattarini <address@hidden>
Date: Mon Aug 8 00:16:56 2011 +0200
parallel-tests: work around Solaris XPG4 make segfault
Issue exposed by test `posixsubst-tests-p.test', and similar to
the problem solved by commit `v1.11-159-ge7aa360'.
* lib/am/check.am [%?PARALLEL_TESTS%] (check-TESTS): Trim trailing
whitespace from $list, to avoid triggering a nasty bug (potential
segfault) on Solaris XPG4 make and Heirloom make.
commit 0ef3ef52079a967a4ef15968fb2054f1b215e7fb
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 7 23:48:02 2011 +0200
testsuite: fix weird spurious failure with Solaris /bin/sh
Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can
apparently end up exiting with exit status 208, instead of leaving
the correct wide exit status to the parent. See:
<http://dbaspot.com/shell/396118-bourne-shell-exit-code-term.html>
Work around this incompatibility.
* tap-signal.test: Look for the above Solaris bug.
(signal_caught): Adapt to handle it.
commit 1072f686df5bb1035620af4a22007feb290d1f12
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 7 23:05:37 2011 +0200
testsuite: fix another spurious failure on Solaris make
* tests/parallel-tests-log-override-recheck.test: Filter make
output before grepping it, for make implementations that, like
Solaris' one, print the whole of the failed recipe on failure.
commit b7e96983128caec94b256760ae9694c262775988
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 7 22:50:23 2011 +0200
testsuite: fix two spurious failures on Solaris make
* tests/tests-environment-and-log-compiler.test: Do not fail if
"make -n check" fails, as that is issued only for debugging, its
semantics being tested in other test scripts.
* tests/test-driver-strip-vpath.test: Likewise.
commit 42851b794a9bed9fdbb7240b551ebd6bd55f77e7
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 7 21:34:56 2011 +0200
tap driver: refactor and remove dead code
* lib/tap-driver (get_tap_line): Removed, ...
(all callers): ... updated to use $parser->next directly instead.
(peek_tap_line, unget_tap_line, @tap_lines): Remove, they're not
used anymore.
(TAP_PEEKING): Block deleted, all its subroutines and variables
having been removed.
commit 30bef0f3dc2e56b1205e3a1d69f5321a9c7a99e6
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 7 20:26:34 2011 +0200
testsuite: fix spurious failure in a test on TAP support
* tests/tap-passthrough-exit.test: When the `--ignore-exit' flag
of the TAP driver is used, don't look for a message reporting the
non-zero exit statuses of tests in the log files; such message is
not expected to be there anymore. Related simplifications.
commit 4740f56e6fe3360fc21e5de68600a777fd046e99
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 7 20:07:35 2011 +0200
tap: plan location is more liberal w.r.t. non-TAP lines
With this change, only lines that are TAP results will matter
w.r.t. the position of the TAP plan in the input; for example,
this input:
this is a non-TAP line
# and this a TAP diagnostic line
1..1
ok 1
was considered to be an error, diagnosed with a message "test
plan in middle of output"; as effect of the current change, such
input is now valid. This is more consistent with the behaviour
of the `prove' utility.
* lib/tap-driver ($lineno): Removed, no more needed.
($tap_stopped): New global variable.
(stringify_test_result): Return "ERROR" if a TAP result is found
when `$tap_stopped' is set to true.
(handle_tap_test): Diagnose TAP results that comes after a late
plan. Add a couple of blank lines, for clarity.
(handle_tap_plan): Set `$tap_stopped' to true after a late plan
is encountered. Do not complain anymore for extra non-TAP lines
preceding or following the plan. Adjust comments.
(main): Don't increment $lineno anymore.
* tests/tap-plan.test: Extend a bit, and remove stale comment.
* tests/tap-color.test: Adjust.
* tests/tap-passthrough.test: Likewise.
* tests/tap-plan-corner.test: Adjust and extend.
* tests/tap-plan-errors.test: Likewise.
* tests/tap-plan-middle.test: New test.
* tests/tap-plan-corner2.test: Delete, it's obsolete now.
* tests/Makefile.am (XFAIL_TESTS): Remove it.
(tap_with_common_setup_tests): Likewise, and add
`tap-plan-corner.test'.
commit e0da0d0740c8174270dae2eb2d5fc1f3e8c6c041
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 7 18:17:27 2011 +0200
testsuite: remove now-passing test from XFAIL_TESTS
* tests/Makefile.am (XFAIL_TESTS): Remove `tap-signal.test',
which is passing since previous commit `v1.11-974-gc7fa872'.
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 109 ++++++++++++++++++++
automake.in | 2 +-
lib/Automake/tests/Makefile.in | 1 +
lib/am/check.am | 24 +++-
lib/tap-driver | 57 ++++-------
tests/Makefile.am | 5 +-
tests/Makefile.in | 26 +++--
tests/parallel-tests-log-override-recheck.test | 9 ++
tests/tap-color.test | 8 +-
tests/tap-passthrough-exit.test | 11 +--
tests/tap-passthrough.test | 2 +-
tests/tap-plan-corner.test | 89 ++++++++++++++---
tests/tap-plan-errors.test | 31 +++---
...{tap-plan-corner2.test => tap-plan-middle.test} | 56 ++++++++--
tests/tap-plan.test | 11 +-
tests/tap-signal.test | 25 +++++-
tests/test-driver-strip-vpath.test | 4 +-
...subdir.test => test-harness-vpath-rewrite.test} | 32 ++++--
tests/test-trs-basic.test | 35 +++++--
tests/tests-environment-and-log-compiler.test | 2 +-
20 files changed, 400 insertions(+), 139 deletions(-)
rename tests/{tap-plan-corner2.test => tap-plan-middle.test} (51%)
copy tests/{parallel-tests-subdir.test => test-harness-vpath-rewrite.test}
(64%)
diff --git a/ChangeLog b/ChangeLog
index bd71052..5330296 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,112 @@
+2011-08-08 Stefano Lattarini <address@hidden>
+
+ test harness: work around a VPATH rewrite issue
+ * lib/am/check.am (am__TEST_BASES): Removed, it's role taken
+ over by ...
+ (am__set_TESTS_bases): ... these new variable.
+ ($(TEST_SUITE_LOG): Use it, to avoid VPATH rewrite issues.
+ * automake.in (handle_tests): Update the code for the cleanup
+ of the `.trs' file to use `$(TEST_LOGS)' instead of
+ `$(am__TEST_BASES)'.
+ * tests/test-harness-vpath-rewrite.test: New test, better
+ exposing the problem fixed by this change
+ * tests/Makefile.am (TESTS): Add it.
+ * tests/test-trs-basic.test: Update and extend.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ parallel-tests: work around Solaris XPG4 make segfault
+ Issue exposed by test `posixsubst-tests-p.test', and similar to
+ the problem solved by commit `v1.11-159-ge7aa360'.
+ * lib/am/check.am [%?PARALLEL_TESTS%] (check-TESTS): Trim trailing
+ whitespace from $list, to avoid triggering a nasty bug (potential
+ segfault) on Solaris XPG4 make and Heirloom make.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ testsuite: fix weird spurious failure with Solaris /bin/sh
+ Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can
+ apparently end up exiting with exit status 208, instead of leaving
+ the correct wide exit status to the parent. See:
+ <http://dbaspot.com/shell/396118-bourne-shell-exit-code-term.html>
+ Work around this incompatibility.
+ * tap-signal.test: Look for the above Solaris bug.
+ (signal_caught): Adapt to handle it.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ testsuite: fix another spurious failure on Solaris make
+ * tests/parallel-tests-log-override-recheck.test: Filter make
+ output before grepping it, for make implementations that, like
+ Solaris' one, print the whole of the failed recipe on failure.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ testsuite: fix two spurious failures on Solaris make
+ * tests/tests-environment-and-log-compiler.test: Do not fail if
+ "make -n check" fails, as that is issued only for debugging, its
+ semantics being tested in other test scripts.
+ * tests/test-driver-strip-vpath.test: Likewise.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ tap driver: refactor and remove dead code
+ * lib/tap-driver (get_tap_line): Removed, ...
+ (all callers): ... updated to use $parser->next directly instead.
+ (peek_tap_line, unget_tap_line, @tap_lines): Remove, they're not
+ used anymore.
+ (TAP_PEEKING): Block deleted, all its subroutines and variables
+ having been removed.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ testsuite: fix spurious failure in a test on TAP support
+ * tests/tap-passthrough-exit.test: When the `--ignore-exit' flag
+ of the TAP driver is used, don't look for a message reporting the
+ non-zero exit statuses of tests in the log files; such message is
+ not expected to be there anymore. Related simplifications.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ tap: plan location is more liberal w.r.t. non-TAP lines
+ With this change, only lines that are TAP results will matter
+ w.r.t. the position of the TAP plan in the input; for example,
+ this input:
+ this is a non-TAP line
+ # and this a TAP diagnostic line
+ 1..1
+ ok 1
+ was considered to be an error, diagnosed with a message "test
+ plan in middle of output"; as effect of the current change, such
+ input is now valid. This is more consistent with the behaviour
+ of the `prove' utility.
+ * lib/tap-driver ($lineno): Removed, no more needed.
+ ($tap_stopped): New global variable.
+ (stringify_test_result): Return "ERROR" if a TAP result is found
+ when `$tap_stopped' is set to true.
+ (handle_tap_test): Diagnose TAP results that comes after a late
+ plan. Add a couple of blank lines, for clarity.
+ (handle_tap_plan): Set `$tap_stopped' to true after a late plan
+ is encountered. Do not complain anymore for extra non-TAP lines
+ preceding or following the plan. Adjust comments.
+ (main): Don't increment $lineno anymore.
+ * tests/tap-plan.test: Extend a bit, and remove stale comment.
+ * tests/tap-color.test: Adjust.
+ * tests/tap-passthrough.test: Likewise.
+ * tests/tap-plan-corner.test: Adjust and extend.
+ * tests/tap-plan-errors.test: Likewise.
+ * tests/tap-plan-middle.test: New test.
+ * tests/tap-plan-corner2.test: Delete, it's obsolete now.
+ * tests/Makefile.am (XFAIL_TESTS): Remove it.
+ (tap_with_common_setup_tests): Likewise, and add
+ `tap-plan-corner.test'.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
+ testsuite: remove now-passing test from XFAIL_TESTS
+ * tests/Makefile.am (XFAIL_TESTS): Remove `tap-signal.test',
+ which is passing since previous commit `v1.11-974-gc7fa872'.
+
2011-08-06 Stefano Lattarini <address@hidden>
tap driver: handle signals received by the tests being run
diff --git a/automake.in b/automake.in
index bccacf1..b8fdde3 100644
--- a/automake.in
+++ b/automake.in
@@ -5119,7 +5119,7 @@ sub handle_tests
}
}
$clean_files{'$(TEST_LOGS)'} = MOSTLY_CLEAN;
- $clean_files{'$(am__TEST_BASES:=.trs)'} = MOSTLY_CLEAN;
+ $clean_files{'$(TEST_LOGS:.log=.trs)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_SUITE_LOG)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_SUITE_HTML)'} = MOSTLY_CLEAN;
}
diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index ad08b66..fc0ccf8 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -461,6 +461,7 @@ check-TESTS:
list=`for f in $$list; do \
test .log = $$f || echo $$f; \
done | tr '\012\015' ' '`; \
+ list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_LOG) TEST_LOGS="$$list"
.log.html:
diff --git a/lib/am/check.am b/lib/am/check.am
index 8340850..035b4c1 100644
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -104,10 +104,19 @@ case " $(XFAIL_TESTS) " in
\
esac; \
$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
-# The names of the tests, with any registered extension removed. Or
-# equivalently, the names of the test logs, with the `.log' extension
-# renoved. This honours runtime overriding of TESTS and TEST_LOGS.
-am__TEST_BASES = $(TEST_LOGS:.log=)
+# A shell command to get the names of the tests scripts with any registered
+# extension removed (i.e., equivalently, the names of the test logs, with
+# the `.log' extension removed). The result is saved in the shell variable
+# `$bases'. This honors runtime overriding of TESTS and TEST_LOGS. Sadly,
+# we cannot use something simpler, involving e.g., "$(TEST_LOGS:.log=)",
+# since that might cause problem with VPATH rewrites for suffix-less tests.
+# See also 'test-harness-vpath-rewrite.test'.
+am__set_TESTS_bases = \
+ bases='$(TEST_LOGS)'; \
+ bases=`for f in $$bases; do echo $$f; done | sed 's/\.log//'`; \
+## Trim away any extra whitespace. This has already proved useful in
+## avoiding weird bug on lesser make implementations.
+ bases=`echo $$bases`
# This can be used instead of $(MAKE) in recipes requiring a recursive call
# to make, but which are not intended to be executed by "make -n". See the
@@ -125,7 +134,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
## Helper shell function, tells whether a path refers to an existing,
## regular, readable file.
f_ok () { test -f "$$1" && test -r "$$1"; }; \
- bases='$(am__TEST_BASES)'; \
+ $(am__set_TESTS_bases); \
ws='[ ]'; \
## We need to ensures that all the required `.trs' and `.log' files will
## be present and readable. The direct dependencies of $(TEST_SUITE_LOG)
@@ -294,6 +303,9 @@ check-TESTS:
list=`for f in $$list; do \
test .log = $$f || echo $$f; \
done | tr '\012\015' ' '`; \
+## This apparently useless munging helps to avoid a nasty bug (a
+## segmentation fault!) on Solaris XPG4 make.
+ list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_LOG) TEST_LOGS="$$list"
AM_RECURSIVE_TARGETS += check
@@ -340,7 +352,7 @@ AM_RECURSIVE_TARGETS += check-html
recheck recheck-html:
@ws='[ ]'; \
target=`echo $@ | sed 's,^re,,'`; \
- bases='$(am__TEST_BASES)'; \
+ $(am__set_TESTS_bases); \
list=`for b in $$bases; do \
## Skip tests that haven't been run, but recover gracefully from deleted
## `.trs' files.
diff --git a/lib/tap-driver b/lib/tap-driver
index 1f5a271..2393346 100755
--- a/lib/tap-driver
+++ b/lib/tap-driver
@@ -43,11 +43,14 @@ my %COLOR = (
# Global variables. #
# ------------------- #
-my $lineno = 0; # Number of input lines seen so far.
my $testno = 0; # Number of test results seen so far.
my $plan_seen = 0; # Whether the TAP plan has been seen or not.
my $parser; # TAP parser object (will be initialized later).
+# When true, it means that the rest of the input stream cannot
+# contain any further TAP results.
+my $tap_stopped = 0;
+
# ----------------- #
# Option parsing. #
# ----------------- #
@@ -95,7 +98,6 @@ sub decorate_result ($);
sub extract_tap_comment ($);
sub finish ();
sub get_global_test_result ();
-sub get_tap_line ();
sub get_test_exit_message ();
sub get_test_results ();
sub handle_tap_bailout ($);
@@ -103,12 +105,10 @@ sub handle_tap_plan ($);
sub handle_tap_test ($);
sub main (@);
sub must_recheck ();
-sub peek_tap_line ();
sub report ($;$);
sub start (@);
sub stringify_test_result ($);
sub testsuite_error ($);
-sub unget_tap_line ($);
sub write_test_results ();
sub yn ($);
@@ -140,29 +140,6 @@ sub yn ($)
return $bool ? "yes" : "no";
}
-TAP_PEEKING :
-{
- my @tap_lines = ();
-
- sub get_tap_line ()
- {
- @tap_lines > 0 ? pop @tap_lines : $parser->next;
- }
-
- sub unget_tap_line ($)
- {
- push @tap_lines, @_;
- }
-
- sub peek_tap_line ()
- {
- my $res = get_tap_line;
- unget_tap_line ($res);
- return $res;
- }
-
-}
-
TEST_RESULTS :
{
my (@test_results, %test_results);
@@ -235,7 +212,7 @@ sub get_test_exit_message ()
{
# Flush all the remaining TAP stream, so that we can obtain the
# exit status of the TAP producer.
- do {} while defined get_tap_line ();
+ do {} while defined $parser->next;
my $wstatus = $parser->wait;
# Return an undefined value if the producer exited with success.
return unless $wstatus;
@@ -272,7 +249,7 @@ sub stringify_test_result ($)
my $result = shift;
my $PASS = $cfg{"expect-failure"} ? "XPASS": "PASS";
my $FAIL = $cfg{"expect-failure"} ? "XFAIL": "FAIL";
- if ($result->is_unplanned || $result->number != $testno)
+ if ($result->is_unplanned || $result->number != $testno || $tap_stopped)
{
return "ERROR";
}
@@ -362,7 +339,12 @@ sub handle_tap_test ($)
{
$string .= " $description";
}
- if ($test->is_unplanned)
+
+ if ($tap_stopped)
+ {
+ $string .= " # AFTER LATE PLAN";
+ }
+ elsif ($test->is_unplanned)
{
$string .= " # UNPLANNED";
}
@@ -378,20 +360,20 @@ sub handle_tap_test ($)
$string .= " $explanation";
}
}
+
report $test_result, $string;
}
sub handle_tap_plan ($)
{
my $plan = shift;
+ # Only one plan per stream is acceptable.
testsuite_error "multiple test plans" if $plan_seen;
$plan_seen = 1;
- # TAP plan must be either in the first or in the last line.
- if ($lineno > 1 && peek_tap_line)
- {
- testsuite_error "test plan in middle of output";
- return;
- }
+ # TAP plan must come either before or after *all* the TAP results.
+ # So, if we find it after having already seen at least one TAP result,
+ # set a flag signaling that no more TAP results are acceptable.
+ $tap_stopped = 1 if $testno >= 1;
# Nothing more to do, unless the plan contains a SKIP directive.
return
if not defined $plan->directive && length ($plan->directive) > 0;
@@ -424,11 +406,10 @@ sub main (@)
{
start @_;
- while (defined (my $cur = get_tap_line))
+ while (defined (my $cur = $parser->next))
{
# Verbatim copy any input line into the log file.
print $cur->raw . "\n";
- $lineno++;
if ($cur->is_plan)
{
handle_tap_plan ($cur);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 4e7cbda..57d46f4 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,9 +31,7 @@ gcj6.test \
override-conditional-2.test \
pr8365-remake-timing.test \
yacc-dist-nobuild-subdir.test \
-tap-plan-corner2.test \
tap-message-0.test \
-tap-signal.test \
txinfo5.test
@@ -764,6 +762,7 @@ test-driver-create-log-dir.test \
test-driver-strip-vpath.test \
test-driver-trs-suffix-registered.test \
test-driver-fail.test \
+test-harness-vpath-rewrite.test \
test-log.test \
test-metadata-global-log.test \
test-metadata-global-result.test \
@@ -1170,8 +1169,8 @@ tap-passthrough.test \
tap-passthrough-exit.test \
tap-plan.test \
tap-plan-corner.test \
-tap-plan-corner2.test \
tap-plan-errors.test \
+tap-plan-middle.test \
tap-realtime.test \
tap-recheck-logs.test \
tap-skip-whole-whitespace.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 12ba08f..fccee03 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -160,10 +160,17 @@ case " $(XFAIL_TESTS) " in
\
am__expect_failure=no;; \
esac; \
$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
-# The names of the tests, with any registered extension removed. Or
-# equivalently, the names of the test logs, with the `.log' extension
-# renoved. This honours runtime overriding of TESTS and TEST_LOGS.
-am__TEST_BASES = $(TEST_LOGS:.log=)
+# A shell command to get the names of the tests scripts with any registered
+# extension removed (i.e., equivalently, the names of the test logs, with
+# the `.log' extension removed). The result is saved in the shell variable
+# `$bases'. This honors runtime overriding of TESTS and TEST_LOGS. Sadly,
+# we cannot use something simpler, involving e.g., "$(TEST_LOGS:.log=)",
+# since that might cause problem with VPATH rewrites for suffix-less tests.
+# See also 'test-harness-vpath-rewrite.test'.
+am__set_TESTS_bases = \
+ bases='$(TEST_LOGS)'; \
+ bases=`for f in $$bases; do echo $$f; done | sed 's/\.log//'`; \
+ bases=`echo $$bases`
# This can be used instead of $(MAKE) in recipes requiring a recursive call
# to make, but which are not intended to be executed by "make -n". See the
# GNU make manual for more details.
@@ -285,8 +292,7 @@ EXTRA_DIST = ChangeLog-old gen-parallel-tests
instspc-tests.sh \
extract-testsuite-summary tap-setup.sh tap-summary-aux.sh
XFAIL_TESTS = all.test auxdir2.test cond17.test gcj6.test \
override-conditional-2.test pr8365-remake-timing.test \
- yacc-dist-nobuild-subdir.test tap-plan-corner2.test \
- tap-message-0.test tap-signal.test txinfo5.test \
+ yacc-dist-nobuild-subdir.test tap-message-0.test txinfo5.test \
$(instspc_xfail_tests)
parallel_tests = backcompat5-p.test check-exported-srcdir-p.test \
check-fd-redirect-p.test check-tests-in-builddir-p.test \
@@ -1018,6 +1024,7 @@ test-driver-create-log-dir.test \
test-driver-strip-vpath.test \
test-driver-trs-suffix-registered.test \
test-driver-fail.test \
+test-harness-vpath-rewrite.test \
test-log.test \
test-metadata-global-log.test \
test-metadata-global-result.test \
@@ -1406,8 +1413,8 @@ tap-passthrough.test \
tap-passthrough-exit.test \
tap-plan.test \
tap-plan-corner.test \
-tap-plan-corner2.test \
tap-plan-errors.test \
+tap-plan-middle.test \
tap-realtime.test \
tap-recheck-logs.test \
tap-skip-whole-whitespace.test \
@@ -1498,7 +1505,7 @@ cscope cscopelist:
$(TEST_SUITE_LOG): $(TEST_LOGS)
@$(am__sh_e_setup); $(am__tty_colors); \
f_ok () { test -f "$$1" && test -r "$$1"; }; \
- bases='$(am__TEST_BASES)'; \
+ $(am__set_TESTS_bases); \
ws='[ ]'; \
redo_bases=`for b in $$bases; do \
f_ok $$b.trs && f_ok $$b.log || echo $$b; \
@@ -1615,6 +1622,7 @@ check-TESTS:
list=`for f in $$list; do \
test .log = $$f || echo $$f; \
done | tr '\012\015' ' '`; \
+ list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_LOG) TEST_LOGS="$$list"
.log.html:
@@ -1644,7 +1652,7 @@ check-html:
recheck recheck-html:
@ws='[ ]'; \
target=`echo $@ | sed 's,^re,,'`; \
- bases='$(am__TEST_BASES)'; \
+ $(am__set_TESTS_bases); \
list=`for b in $$bases; do \
test -f $$b.trs || test -f $$b.log || continue; \
grep "^$$ws*:recheck:$$ws*no$$ws*$$" $$b.trs \
diff --git a/tests/parallel-tests-log-override-recheck.test
b/tests/parallel-tests-log-override-recheck.test
index db73718..da3bcd2 100755
--- a/tests/parallel-tests-log-override-recheck.test
+++ b/tests/parallel-tests-log-override-recheck.test
@@ -56,6 +56,13 @@ $ACLOCAL
$AUTOCONF
$AUTOMAKE -a
+# Filter make output before grepping it, for make implementations that,
+# like Solaris' one, print the whole of the failed recipe on failure.
+filter_stdout ()
+{
+ grep -v 'bases=.*;' stdout > t && mv -f t stdout
+}
+
./configure
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
@@ -65,6 +72,7 @@ TEST_SUITE_LOG=my.log $MAKE -e recheck >stdout \
&& { cat stdout; Exit 1; }
cat stdout
ls -l
+filter_stdout
count_test_results total=2 pass=0 fail=1 skip=0 xfail=0 xpass=0 error=1
for x in stdout my.log; do
$FGREP foofoo $x && Exit 1
@@ -78,6 +86,7 @@ BAZ_EXIT_STATUS=0 TEST_SUITE_LOG=my2.log $MAKE -e recheck
>stdout \
cat stdout
ls -l
count_test_results total=2 pass=1 fail=0 skip=0 xfail=0 xpass=0 error=1
+filter_stdout
$FGREP foofoo stdout && Exit 1
$FGREP barbar stdout
$FGREP bazbaz stdout
diff --git a/tests/tap-color.test b/tests/tap-color.test
index dbd6629..cd0329b 100755
--- a/tests/tap-color.test
+++ b/tests/tap-color.test
@@ -70,9 +70,10 @@ Bail out!
END
cat > badplan.test << 'END'
-# foo
-1..1
+1..2
ok 1
+1..2
+ok 2
END
cat > noplan.test << 'END'
@@ -109,7 +110,8 @@ test_color ()
cat stdout | grep "^${blu}SKIP${std}: skip\.test - whole script$"
cat stdout | grep "^${grn}PASS${std}: bail\.test 1$"
cat stdout | grep "^${mgn}ERROR${std}: bail\.test - Bail out!$"
- cat stdout | grep "^${mgn}ERROR${std}: badplan\.test - test plan in middle
of output$"
+ cat stdout | grep "^${mgn}ERROR${std}: badplan\.test - multiple test plans$"
+ cat stdout | grep "^${mgn}ERROR${std}: badplan\.test 2 # AFTER LATE PLAN$"
cat stdout | grep "^${mgn}ERROR${std}: noplan\.test - missing test plan$"
cat stdout | grep "^${mgn}ERROR${std}: few.test - too few tests run
(expected 2, got 1)$"
cat stdout | grep "^${mgn}ERROR${std}: many.test - too many tests run
(expected 1, got 2)$"
diff --git a/tests/tap-passthrough-exit.test b/tests/tap-passthrough-exit.test
index ae8ffa8..68d7071 100755
--- a/tests/tap-passthrough-exit.test
+++ b/tests/tap-passthrough-exit.test
@@ -53,14 +53,7 @@ for e in $exit_statuses; do
done
done
-st=0
-env TEST_LOG_DRIVER_FLAGS='--ignore-exit' $MAKE -e check || st=$?
-for e in $exit_statuses; do cat exit-$e.log; done
-cat test-suite.log
-test $st -eq 0 || Exit 1
-
-for e in $exit_statuses; do
- grep "exit-$e\\.test - exited with status $e$" exit-$e.log
-done
+env TEST_LOG_DRIVER_FLAGS='--ignore-exit' $MAKE -e check
+$FGREP ".test - exited with status" *.log && Exit 1
:
diff --git a/tests/tap-passthrough.test b/tests/tap-passthrough.test
index ba773ed..6406548 100755
--- a/tests/tap-passthrough.test
+++ b/tests/tap-passthrough.test
@@ -154,7 +154,7 @@ for rx in \
'^ok 23$' \
'^Misplaced plan$' \
'^1\.\.13$' \
- '^ERROR:.* test plan in middle of output' \
+ '^ERROR:.* multiple test plans' \
'^Extra test$' \
'^Last line$' \
'^ERROR:.* [tT]oo many tests run.*expected 3, got 4' \
diff --git a/tests/tap-plan-corner.test b/tests/tap-plan-corner.test
index 4215556..eaebcf2 100755
--- a/tests/tap-plan-corner.test
+++ b/tests/tap-plan-corner.test
@@ -22,42 +22,103 @@ parallel_tests=yes
. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
-cat > leading-repeated-1.test <<END
+cat > leading-repeated.test <<END
1..1
1..1
ok 1
END
-cat > trailing-repeated-1.test <<END
+cat > trailing-repeated.test <<END
ok 1
1..1
1..1
END
-cat > leading-repeated-2.test <<END
+for pos in leading trailing; do
+ TESTS="$pos-repeated.test" $MAKE -e check >stdout \
+ && { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1
+ grep "^ERROR: $pos-repeated\\.test - multiple test plans$" stdout
+done
+
+cat > leading-repeated.test <<END
1..2
ok 1
1..2
ok 2
END
-cat > trailing-repeated-2.test <<END
+cat > trailing-repeated.test <<END
ok 1
1..2
ok 2
1..2
END
-for i in 1 2; do
- for kind in leading trailing; do
- cp $kind-repeated-$i.test all.test
- $MAKE check >stdout && { cat stdout; Exit 1; }
- cat stdout
- count_test_results total=`expr $i + 2` pass=$i fail=0 \
- xpass=0 xfail=0 skip=0 error=2
- grep '^ERROR: all\.test - test plan in middle of output$' stdout
- grep '^ERROR: all\.test - multiple test plans$' stdout
- done
+env TESTS="leading-repeated.test trailing-repeated.test" \
+ $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=6 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=4
+for pos in leading trailing; do
+ grep "^ERROR: $pos-repeated\\.test - multiple test plans$" stdout
+ grep "^ERROR: $pos-repeated\\.test 2 # AFTER LATE PLAN$" stdout
done
+cat > all.test <<END
+1..5
+ok 1
+ok 2
+1..5
+ok 3
+1..5
+ok 4
+1..5
+ok 5
+END
+
+$MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=8 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=6
+
+cat > exp <<'END'
+PASS: all.test 1
+PASS: all.test 2
+ERROR: all.test - multiple test plans
+ERROR: all.test 3 # AFTER LATE PLAN
+ERROR: all.test - multiple test plans
+ERROR: all.test 4 # AFTER LATE PLAN
+ERROR: all.test - multiple test plans
+ERROR: all.test 5 # AFTER LATE PLAN
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+sed -e 1d all.test > t
+mv -f t all.test
+
+$MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=7 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=5
+
+cat > exp <<'END'
+PASS: all.test 1
+PASS: all.test 2
+ERROR: all.test 3 # AFTER LATE PLAN
+ERROR: all.test - multiple test plans
+ERROR: all.test 4 # AFTER LATE PLAN
+ERROR: all.test - multiple test plans
+ERROR: all.test 5 # AFTER LATE PLAN
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
:
diff --git a/tests/tap-plan-errors.test b/tests/tap-plan-errors.test
index fd12e1b..95cc640 100755
--- a/tests/tap-plan-errors.test
+++ b/tests/tap-plan-errors.test
@@ -16,9 +16,9 @@
# TAP support: the following situations should be flagged as errors:
# - unmatched test plan (too few tests run)
-# - misplaced test plan
# - multiple test plans
# - missing test plan
+# - misplaced test plan (tests run after a late plan)
# Checks about unplanned tests are performed in 'tap-unplanned.test'.
# More checks about corner-cases in TAP plans are performed in
# 'tap-plan-corner.test' and 'tap-plan-corner2.test'.
@@ -36,17 +36,17 @@ my_check ()
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
count_test_results "$@"
- grep "^ERROR: all\\.test - $err$" stdout
+ grep "^ERROR: all\\.test $err$" stdout
unset err
}
-err='too few tests run (expected 2, got 1)'
+err='- too few tests run (expected 2, got 1)'
my_check total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
1..2
ok 1
END
-err='too few tests run (expected 12, got 3)'
+err='- too few tests run (expected 12, got 3)'
my_check total=4 pass=2 fail=0 xpass=0 xfail=1 skip=0 error=1 <<END
ok 1
ok 2
@@ -54,29 +54,30 @@ not ok 3 # TODO
1..12
END
-err='too few tests run (expected 1, got 0)'
+err='- too few tests run (expected 1, got 0)'
my_check total=1 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
1..1
END
-err='test plan in middle of output'
-my_check total=3 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
-oops
-1..2
+err='2 # AFTER LATE PLAN'
+my_check total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
ok 1
+1..2
ok 2
END
-err='test plan in middle of output'
-my_check total=3 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+err='5 # AFTER LATE PLAN'
+my_check total=5 pass=4 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
ok 1
ok 2
-1..2
-oops
+ok 3
+ok 4
+1..5
+ok 5
END
# The two test plans here are deliberately equal.
-err='multiple test plans'
+err='- multiple test plans'
my_check total=3 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
1..2
ok 1
@@ -84,7 +85,7 @@ ok 2
1..2
END
-err='missing test plan'
+err='- missing test plan'
my_check total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
ok 1
END
diff --git a/tests/tap-plan-corner2.test b/tests/tap-plan-middle.test
similarity index 51%
rename from tests/tap-plan-corner2.test
rename to tests/tap-plan-middle.test
index 54fd5e9..88bb647 100755
--- a/tests/tap-plan-corner2.test
+++ b/tests/tap-plan-middle.test
@@ -15,35 +15,69 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# TAP support:
-# - more corner cases for TAP plan
+# - test plan preceding and/or following non-result TAP lines
parallel_tests=yes
. ./defs || Exit 1
. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
-# The leading blank line is meant.
-cat > leading-blank.test <<END
+cat > top1.test <<END
+non-TAP line, ignored
+1..1
+ok 1
+END
+cat > top2.test <<END
+some
+non-TAP
+lines
+are
+ignored
+# and a TAP comment won't cause problems either
1..2
ok 1
ok 2
END
-# The trailing blank line is meant.
-cat > trailing-blank.test <<END
+# Try with a blank line too, just to be sure.
+cat > top3.test <<END
+
+1..1
+ok 1
+END
+
+cat > bot1.test <<END
+ok 1 # SKIP
+1..1
+bla blah blah ...
+END
+
+cat > bot2.test <<END
ok 1
ok 2
-1..2
+not ok 3 # TODO
+1..3
address@hidden (a cursing comment :-)
+END
+
+# Try with a blank line too, just to be sure.
+cat > bot3.test <<END
+ok 1
+not ok 2 # TODO
+ok 3 # SKIP
+ok 4 # SKIP
+1..4
END
-for kind in leading trailing; do
- cp $kind-blank.test all.test
- $MAKE check >stdout && { cat stdout; Exit 1; }
+tests=`echo *.test`
+
+for tap_flags in "" "--comments"; do
+ env TEST_LOG_DRIVER_FLAGS="$tap_flags" TESTS="$tests" \
+ $MAKE -e check >stdout || { cat stdout; Exit 1; }
cat stdout
- count_test_results total=3 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=1
- grep '^ERROR: all\.test - test plan in middle of output$' stdout
+ count_test_results total=12 pass=7 xfail=2 skip=3 fail=0 xpass=0 error=0
done
:
diff --git a/tests/tap-plan.test b/tests/tap-plan.test
index 79e9bdd..e2fda35 100755
--- a/tests/tap-plan.test
+++ b/tests/tap-plan.test
@@ -17,7 +17,6 @@
# TAP support:
# - test scripts with the test plan at the beginning
# - test scripts with the test plan at the end
-# - test scripts without a test plan
parallel_tests=yes
. ./defs || Exit 1
@@ -42,10 +41,12 @@ ok
1..4
END
-# Check that the plans doesn't cause any problem
+for tap_flags in "" "--comments"; do
+ env TEST_LOG_DRIVER_FLAGS="$tap_flags" TESTS='top.test bot.test' \
+ $MAKE -e check >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=7 pass=5 xfail=1 skip=1 fail=0 xpass=0 error=0
+done
-TESTS='top.test bot.test' $MAKE -e check >stdout || { cat stdout; Exit 1; }
-cat stdout
-count_test_results total=7 pass=5 xfail=1 skip=1 fail=0 xpass=0 error=0
:
diff --git a/tests/tap-signal.test b/tests/tap-signal.test
index ce75ddd..8ceeddd 100755
--- a/tests/tap-signal.test
+++ b/tests/tap-signal.test
@@ -36,13 +36,36 @@ chmod a+x *.test
. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+# Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can end up
+# exiting with exit status 208, instead of leaving the correct wide exit
+# status to the parent. See:
+# <http://dbaspot.com/shell/396118-bourne-shell-exit-code-term.html>
+# We need to detect and work around this incompatibility.
+
+if /bin/sh -c 'kill -2 $$'; then
+ fatal_ "/bin/sh cannot kill itself"
+elif test $? -eq 208; then
+ have_solaris_bug=yes
+else
+ have_solaris_bug=no
+fi
+
signal_caught ()
{
numeric=$1
symbolic=$2
sig_re="((SIG)?$symbolic|$numeric)"
tst_re="signal-$numeric\\.test"
- $EGREP "^ERROR: $tst_re - terminated by signal $sig_re$" stdout
+ if $EGREP "^ERROR: $tst_re - terminated by signal $sig_re$" stdout; then
+ return 0
+ elif test $have_solaris_bug = yes; then
+ case $symbolic in
+ INT|TERM) $EGREP "^ERROR: $tst_re - exited with status 208$" stdout;;
+ *) return 1;;
+ esac
+ else
+ return 1
+ fi
}
all_signals_caught ()
diff --git a/tests/test-driver-strip-vpath.test
b/tests/test-driver-strip-vpath.test
index 5b21cc0..7187abb 100755
--- a/tests/test-driver-strip-vpath.test
+++ b/tests/test-driver-strip-vpath.test
@@ -79,7 +79,7 @@ mkdir build1
cd build1
../src/configure
# "$MAKE -n" is for debugging, should highlight any VPATH rewrite.
-$MAKE -n check
+$MAKE -n check || :
$MAKE check
cd ..
@@ -87,7 +87,7 @@ mkdir build2
cd build2
"$ocwd"/src/configure
# "$MAKE -n" is for debugging, should highlight any VPATH rewrite.
-$MAKE -n check
+$MAKE -n check || :
$MAKE check
cd ..
diff --git a/tests/parallel-tests-subdir.test
b/tests/test-harness-vpath-rewrite.test
similarity index 64%
copy from tests/parallel-tests-subdir.test
copy to tests/test-harness-vpath-rewrite.test
index 879c858..57ff909 100755
--- a/tests/parallel-tests-subdir.test
+++ b/tests/test-harness-vpath-rewrite.test
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2011 Free Software Foundation, Inc.
+# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -14,8 +14,8 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check that the parallel-tests driver creates parent directories for
-# the log files when needed.
+# Check that our concurrent test harness is not subject to spurious VPATH
+# rewrites.
parallel_tests=yes
. ./defs || Exit 1
@@ -25,13 +25,17 @@ AC_OUTPUT
END
cat > Makefile.am << 'END'
-TESTS = dir1/foo.test dir2/dir3/foo.test
-TEST_LOG_COMPILER = sh
+LOG_COMPILER = true
+empty =
+TESTS = x0 foo bar baz x1
+EXTRA_DIST = $(TESTS)
END
-mkdir dir1 dir2 dir2/dir3
-echo : > dir1/foo.test
-echo : > dir2/dir3/foo.test
+: > x0
+: > foo
+: > bar
+: > baz
+: > x1
$ACLOCAL
$AUTOCONF
@@ -41,9 +45,15 @@ mkdir build
cd build
../configure
$MAKE check
-find . # For debugging.
test -f test-suite.log
-test -f dir1/foo.log
-test -f dir2/dir3/foo.log
+test -f foo.log
+test -f foo.trs
+test -f bar.log
+test -f bar.trs
+test -f baz.log
+test -f baz.trs
+# We don't want VPATH rewrites, really.
+$MAKE check -n | $EGREP '\.\./(foo|bar|baz)( |$)' && Exit 1
+$MAKE distcheck
:
diff --git a/tests/test-trs-basic.test b/tests/test-trs-basic.test
index 88f1276..a9c34a5 100755
--- a/tests/test-trs-basic.test
+++ b/tests/test-trs-basic.test
@@ -31,9 +31,10 @@ TESTS = foo.test bar.sh sub/zardoz.test
TEST_LOG_COMPILER = $(SHELL)
SH_LOG_COMPILER = $(SHELL)
-## Used to check some internal details.
+## Used to check some internal details. And yes, the quotes around
+## `$bases' are deliberate: they check for whitespace normalization.
tb:
- echo $(am__TEST_BASES) > $@
+ $(am__set_TESTS_bases); echo "$$bases" > $@
END
cat > foo.test << 'END'
@@ -50,18 +51,34 @@ $ACLOCAL
$AUTOCONF
$AUTOMAKE -a
-./configure
#
# Check some internal details first.
#
-$MAKE tb
-test x"`cat tb`" = x"foo bar sub/zardoz"
-rm -f tb
-TESTS='foo.test foo2.sh foo3.exe foo4' $MAKE -e tb
-test x"`cat tb`" = x"foo foo2 foo3.exe foo4"
-rm -f tb
+for vpath in : false; do
+
+ if $vpath; then
+ srcdir=..
+ mkdir build
+ cd build
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ $MAKE tb
+ test x"`cat tb`" = x"foo bar sub/zardoz"
+ rm -f tb
+ # Please don't change the order of the stuff in TESTS, below.
+ TESTS='a foo.test foo2.sh foo3 foo.log foo-log foolog b.exe' $MAKE -e tb
+ test x"`cat tb`" = x"a foo foo2 foo3 foo.log foo-log foolog b.exe"
+ rm -f tb
+
+ cd $srcdir
+
+done
#
# The `test-suite.stamp' file and the `.trs' files get created by
diff --git a/tests/tests-environment-and-log-compiler.test
b/tests/tests-environment-and-log-compiler.test
index 3753bb9..eafc3bf 100755
--- a/tests/tests-environment-and-log-compiler.test
+++ b/tests/tests-environment-and-log-compiler.test
@@ -65,7 +65,7 @@ $AUTOMAKE -a
./configure
-$MAKE -n check # For debugging.
+$MAKE -n check || : # For debugging.
VERBOSE=yes $MAKE check
cat a.log
cat b.log
hooks/post-receive
--
GNU Automake
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-983-gfda3de5,
Stefano Lattarini <=