[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
parallel-tests: warn about $(srcdir), $(top_srcdir) in TESTS.
From: |
Ralf Wildenhues |
Subject: |
parallel-tests: warn about $(srcdir), $(top_srcdir) in TESTS. |
Date: |
Sat, 28 Mar 2009 22:17:03 +0100 |
User-agent: |
Mutt/1.5.18 (2008-05-17) |
* Ralf Wildenhues wrote on Fri, Mar 20, 2009 at 12:24:15AM CET:
> set x foo*.test ../../source/dir/bar*.test
> shift
> make check "TESTS=$*"
>
> or
> eval make check \'TESTS= foo*.test ../../source/dir/bar*.test \'
Hrmpf. Using $(srcdir) in TESTS is really bad: it causes the test log
to end up in the source tree.
For the above kind of hacks, there are workarounds, like
cd $srcdir; set x foo*.test; shift; cd -
make check TESTS="$*"
We could also rewrite TESTS to eliminate the $(srcdir) prefixes, which
would work for TESTS values passed to 'make', but:
The Makefile.am author might have a good reason to use explicit
$(srcdir) for generated distributed test files, quoting
(autoconf.info)Make Target Lookup:
| OpenBSD and FreeBSD `make', however, never perform a `VPATH' search
| for a dependency that has an explicit rule. This is extremely annoying.
In this case, rewriting TESTS will silently prevent `make' from
considering the rule to update the test. So, let's not do that,
creating silent bugs, but instead error out if we see this.
I'm applying this patch to the ad-parallel-tests branch, also noting
the experimental status of this test driver, we might encounter more
such issues...
BTW, Solaris make VPATH rewriting is not a problem for this particular
issue: it only occurs if TESTS is actually listed as prerequisite to the
rule we're currently executing, but not in TEST_LOGS which is something
like $(TESTS:.test=.log), only more complicated.
Cheers,
Ralf
parallel-tests: warn about $(srcdir), $(top_srcdir) in TESTS.
* automake.in (handle_tests): Warn about portability issue
concerning generated TESTS files listed with a `$(srcdir)/'
or `$(top_srcdir)/' prefix.
* doc/automake.texi (TESTS): Document this issue. Mention that
the parallel-tests driver is still experimental.
* tests/parallel-tests8.test: New test.
* tests/Makefile.am: Update.
diff --git a/automake.in b/automake.in
index 433f88e..ce20cce 100755
--- a/automake.in
+++ b/automake.in
@@ -4789,6 +4789,13 @@ sub handle_tests
return $obj
if $val =~ /address@hidden@$/;
$obj =~ s/\$\(EXEEXT\)$//o;
+
+ if ($val =~ /(\$\((top_)?srcdir\))\//o)
+ {
+ msg ('error', $subvar->rdef ($cond)->location,
+ "parallel-tests: using `$1' in TESTS is currently
broken: `$val'");
+ }
+
foreach my $test_suffix (@test_suffixes)
{
next
diff --git a/doc/automake.texi b/doc/automake.texi
index d3b72a5..bf41acb 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8371,6 +8371,7 @@ by the tests, not the tests themselves. Of course you
can set
@section Simple tests using @samp{parallel-tests}
@cindex @option{parallel-tests}, Using
+
The option @option{parallel-tests} (@pxref{Options}) enables a test
suite driver that is mostly compatible to the simple test driver
described above, but provides a few more features and slightly different
@@ -8383,6 +8384,9 @@ for exceptional failures. Similar to the simple test
driver,
the @code{check_*} variables are honored, and the environment variable
@env{srcdir} is set during test execution.
+This test driver is still experimental and may undergo changes in order
+to satisfy additional portability requirements.
+
@vindex TEST_SUITE_LOG
@vindex TEST_LOGS
The driver operates by defining a set of @command{make} rules to create
@@ -8504,6 +8508,17 @@ the inference rules generated by @command{automake} can
apply. For
literal test names, @command{automake} can generate per-target rules
to avoid this limitation.
+Please note that it is currently not possible to use @code{$(srcdir)/}
+or @code{$(top_srcdir)/} in the @code{TESTS} variable. This technical
+limitation is necessary to avoid generating test logs in the source tree
+and has the unfortunate consequence thast it is not possible to specify
+distributed tests that are themselves generated by means of explicit
+rules, in a way that is portable to all @command{make} implementations
+(@pxref{Make Target Lookup,,, autoconf, The Autoconf Manual}, the
+semantics of FreeBSD and OpenBSD @command{make} conflict with this).
+In case of doubt you may want to require to use GNU @command{make},
+or work around the issue with inference rules to generate the tests.
+
@section DejaGnu Tests
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 3f31ca9..daecf34 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -486,6 +486,7 @@ parallel-tests4.test \
parallel-tests5.test \
parallel-tests6.test \
parallel-tests7.test \
+parallel-tests8.test \
parse.test \
percent.test \
percent2.test \
diff --git a/tests/parallel-tests8.test b/tests/parallel-tests8.test
new file mode 100755
index 0000000..309a2cf
--- /dev/null
+++ b/tests/parallel-tests8.test
@@ -0,0 +1,75 @@
+#! /bin/sh
+# Copyright (C) 2009 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - generated distributed tests.
+# - listing $(srcdir)/ or $(top_srcdir)/ in TESTS doesn't work ATM,
+# and is thus diagnosed.
+
+# TODO: this test should also ensure that the `make' implementation
+# properly adheres to rules in all cases. See the Autoconf
+# manual for the ugliness in this area, when VPATH comes into
+# play. :-/
+
+. ./defs-p || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = foo.test
+## the next line will cause automake to error out:
+TESTS += $(srcdir)/bar.test $(top_srcdir)/baz.test
+.in.test:
+ cp $< $@
+ chmod +x $@
+check_SCRIPTS = $(TESTS)
+EXTRA_DIST = foo.in foo.test
+DISTCLEANFILES = foo.test
+END
+
+cat >>foo.in <<'END'
+#! /bin/sh
+echo "this is $0"
+exit 0
+END
+
+$ACLOCAL
+$AUTOCONF
+AUTOMAKE_fails -a
+grep '(srcdir.*bar' stderr
+grep 'top_srcdir.*baz' stderr
+
+sed '/srcdir/d' < Makefile.am > t
+mv -f t Makefile.am
+$AUTOMAKE -a
+
+./configure
+$MAKE check
+$MAKE distcheck
+$MAKE distclean
+
+mkdir build
+cd build
+../configure
+$MAKE check
+test ! -f ../foo.log
+$MAKE distcheck
+
+:
- [PATCH 09] Mark check-html and check-TESTS as recursive for BSD make., (continued)
- [PATCH 09] Mark check-html and check-TESTS as recursive for BSD make., Ralf Wildenhues, 2009/03/14
- [PATCH 10] Fix parallel-tests with empty $(TESTS) for BSD make., Ralf Wildenhues, 2009/03/14
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Ralf Wildenhues, 2009/03/14
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Akim Demaille, 2009/03/14
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Akim Demaille, 2009/03/14
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Ralf Wildenhues, 2009/03/14
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Akim Demaille, 2009/03/14
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Ralf Wildenhues, 2009/03/17
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Akim Demaille, 2009/03/18
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Ralf Wildenhues, 2009/03/19
- parallel-tests: warn about $(srcdir), $(top_srcdir) in TESTS.,
Ralf Wildenhues <=
- Re: parallel-tests: warn about $(srcdir), $(top_srcdir) in TESTS., Akim Demaille, 2009/03/30
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Akim Demaille, 2009/03/30
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Ralf Wildenhues, 2009/03/30
- Re: [PATCH 11] Fix LAZY_TEST_SUITE handling and $(TEST_SUITE_LOG) recreation., Ralf Wildenhues, 2009/03/22
- check-html: Always create HTML output, note conversion failure., Ralf Wildenhues, 2009/03/28
- parallel-tests: redo lazy checking: recheck and RECHECK_LOGS., Ralf Wildenhues, 2009/03/28
- Re: parallel-tests: redo lazy checking: recheck and RECHECK_LOGS., Akim Demaille, 2009/03/30
- Re: parallel-tests: redo lazy checking: recheck and RECHECK_LOGS., Ralf Wildenhues, 2009/03/30
proposed patch: parallel-tests: also record logs of SKIPped tests., Ralf Wildenhues, 2009/03/14