automake-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[FYI] {testsuite-work} maintcheck: consistency of list of test scripts


From: Stefano Lattarini
Subject: [FYI] {testsuite-work} maintcheck: consistency of list of test scripts
Date: Tue, 3 May 2011 19:11:07 +0200
User-agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; )

Reference:
 <http://lists.gnu.org/archive/html/automake-patches/2011-04/msg00141.html>

On Tuesday 19 April 2011, Stefano Lattarini wrote:
> Wow, this is an oooold one.  References:
>  <http://lists.gnu.org/archive/html/automake-patches/2009-12/msg00026.html>
>  <http://lists.gnu.org/archive/html/automake-patches/2009-12/msg00045.html>
> 
> -*-*-
> 
> maintcheck: consistency of list of test scripts
> 
> * CheckListOfTests.am: New file.
> (maintainer-check-list-of-tests): New target,
> check for consistency between list of tests defined in the
> including Makefile and list of tests on the filesystem.
> (clean-maintcheck-testslist-tmp): New rule, to clean up the
> temporary files that might be left around by the rules associated
> with the previous target.
> (clean-local): Depend on it.
> * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
> (maintainer-check): Added dependency from the new target
> `maintainer-check-list-of-tests'.
> * tests/Makefile.am: Likewise.  Also ...
> (TEST_EXTENSIONS): New, define to `.test'.
> ---
>  ChangeLog                      |   21 ++++++++++++
>  CheckListOfTests.am            |   70 
> ++++++++++++++++++++++++++++++++++++++++
>  Makefile.am                    |   21 +++++++++++-
>  Makefile.in                    |   18 ++++++++++-
>  lib/Automake/tests/Makefile.am |    5 ++-
>  lib/Automake/tests/Makefile.in |   68 ++++++++++++++++++++++++++++++++------
>  tests/Makefile.am              |    5 +++
>  tests/Makefile.in              |   52 ++++++++++++++++++++++++++++--
>  8 files changed, 243 insertions(+), 17 deletions(-)
>  create mode 100644 CheckListOfTests.am
> 
> -*-*-
> 
> OK for maint?  I'll push in 72 hours if there is no objection.
> 
> Regards,
>   Stefano
> 

OK, I had to revise this patch a bit now that in the 'testsuite-work'
branch we have stuff in $(TESTS) that doesn't correspond to existing
files (by a conscious choice).  The updated patch, still based of off
maint, is attached.

I've merged in into 'testsuite-work' and pushed.  I hope it will be
accepted also in maint some not-too-distant day.

Regards,
  Stefano
From 7b6ab07442b28a62389fc4dc8f35276f31348c13 Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Sat, 30 Jan 2010 01:23:44 +0100
Subject: [PATCH] maintcheck: consistency of list of test scripts

* CheckListOfTests.am: New file.
(maintainer-check-list-of-tests): New target,
check for consistency between list of tests defined in the
including Makefile and list of tests on the filesystem.
(clean-maintcheck-testslist-tmp): New rule, to clean up the
temporary files that might be left around by the rules associated
with the previous target.
(clean-local): Depend on it.
* lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
(maintainer-check): Added dependency from the new target
`maintainer-check-list-of-tests'.
* tests/Makefile.am: Likewise.
* Makefile.am (maintainer-check-list-of-test): New target,
calling recursively into `tests/' and `lib/Automake/tests/',
using ...
(TEST_SUBDIRS): ... this new variable.
---
 ChangeLog                      |   20 ++++++++++
 CheckListOfTests.am            |   76 ++++++++++++++++++++++++++++++++++++++++
 Makefile.am                    |   21 ++++++++++-
 Makefile.in                    |   18 +++++++++-
 lib/Automake/tests/Makefile.am |    5 ++-
 lib/Automake/tests/Makefile.in |   70 +++++++++++++++++++++++++++++++------
 tests/Makefile.am              |    3 ++
 tests/Makefile.in              |   52 ++++++++++++++++++++++++++-
 8 files changed, 249 insertions(+), 16 deletions(-)
 create mode 100644 CheckListOfTests.am

diff --git a/ChangeLog b/ChangeLog
index 99e4f11..846b569 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,23 @@
+2011-05-03  Stefano Lattarini  <address@hidden>
+
+       maintcheck: consistency of list of test scripts
+       * CheckListOfTests.am: New file.
+       (maintainer-check-list-of-tests): New target,
+       check for consistency between list of tests defined in the
+       including Makefile and list of tests on the filesystem.
+       (clean-maintcheck-testslist-tmp): New rule, to clean up the
+       temporary files that might be left around by the rules associated
+       with the previous target.
+       (clean-local): Depend on it.
+       * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
+       (maintainer-check): Added dependency from the new target
+       `maintainer-check-list-of-tests'.
+       * tests/Makefile.am: Likewise.
+       * Makefile.am (maintainer-check-list-of-test): New target,
+       calling recursively into `tests/' and `lib/Automake/tests/',
+       using ...
+       (TEST_SUBDIRS): ... this new variable.
+
 2011-04-24  Stefano Lattarini  <address@hidden>
 
        tests: fix spurious failure in a test on TESTS (VPATH-related)
diff --git a/CheckListOfTests.am b/CheckListOfTests.am
new file mode 100644
index 0000000..28cb795
--- /dev/null
+++ b/CheckListOfTests.am
@@ -0,0 +1,76 @@
+## -*- Automake -*-
+##
+## Copyright (C) 2011 Free Software Foundation, Inc.
+##
+## This program is free software; you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation; either version 2, or (at your option)
+## any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program.  If not, see <http://www.gnu.org/licenses/>.
+##
+
+## Temporary files used in the `check-list-of-tests' target.
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
+
+## Check that the list of tests given in the Makefile is equal to the
+## list of all test scripts in the Automake testsuite.
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+## Prefer unified diffs over plain diffs, for readability.
+        @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+          diff='diff -u'; \
+        else \
+          diff='diff'; \
+        fi; \
+## Determine which files are to be considered test scripts.
+## Do not blindly look for `$(TEST_EXTENSIONS)', because the parallel
+## testsuite driver provided by Automake allow entries in `$(TESTS)'
+## not to be real files, but rather paramaters for the corresponding
+## LOG_COMPILER (well, sort of).
+        set X $(checked_test_extensions); shift; \
+        if test $$# -eq 0; then \
+          set X $(TEST_EXTENSIONS); shift; \
+        fi; \
+        if test $$# -eq 0; then \
+          echo '$@: internal error: $$(checked_test_extensions) and' \
+               '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+          exit 255; \
+        fi; \
+## List of tests in Makefile.
+        lst='$(expected_list_of_tests)'; \
+        test -n "$$lst" || lst='$(TESTS)'; \
+        for t in $$lst; do \
+          echo "$$t"; \
+        done | sort >$(am__tmk); \
+## List of tests on filesystem.  Be careful to cater for VPATH builds too.
+        for ext in $$*; do \
+          ls *$$ext 2>/dev/null; \
+          if test $(srcdir) != $(builddir); then \
+            (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+          fi; \
+        done | sort | uniq >$(am__tfs); \
+## Compare the two lists, complain if they differ.
+        if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+           result=0; \
+        else \
+           echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+           echo "+ $$diff in-makefile on-filesystem" >&2; \
+           cat $(am__tdf) >&2; \
+           result=1; \
+        fi; \
+        rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+        exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+       rm -f $(am__tmk) $(am__tfs) $(am__tdf)
diff --git a/Makefile.am b/Makefile.am
index 6e8b7df..aeff613 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -24,6 +24,8 @@
 ## run aclocal and automake.
 SUBDIRS = lib . doc m4 tests
 
+TEST_SUBDIRS = tests lib/Automake/tests
+
 bin_SCRIPTS = automake aclocal
 
 CLEANFILES = $(bin_SCRIPTS)
@@ -150,7 +152,24 @@ sc_at_in_texi
 
 .PHONY: $(syntax_check_rules)
 $(syntax_check_rules): automake aclocal
-maintainer-check: $(syntax_check_rules)
+
+## Check that the list of tests given in the Makefile is equal to the
+## list of all test scripts in the Automake testsuite.
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+       @failcom='exit 1'; \
+       for f in x $$MAKEFLAGS; do \
+         case $$f in \
+           *=* | --[!k]*);; \
+           *k*) failcom='fail=yes';; \
+         esac; \
+       done; \
+       for subdir in $(TEST_SUBDIRS); do \
+         (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \
+       done; \
+       test -z "$$fail"
+
+maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests
 
 ## Look for test whose names can cause spurious failures when used as
 ## first argument to AC_INIT (chiefly because they might contain an
diff --git a/Makefile.in b/Makefile.in
index 49c35f5..ea63d6b 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -236,6 +236,7 @@ top_build_prefix = @top_build_prefix@
 top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
 SUBDIRS = lib . doc m4 tests
+TEST_SUBDIRS = tests lib/Automake/tests
 bin_SCRIPTS = automake aclocal
 CLEANFILES = $(bin_SCRIPTS)
 AUTOMAKESOURCES = automake.in aclocal.in
@@ -928,7 +929,22 @@ dist-hook:
 
 .PHONY: $(syntax_check_rules)
 $(syntax_check_rules): automake aclocal
-maintainer-check: $(syntax_check_rules)
+
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+       @failcom='exit 1'; \
+       for f in x $$MAKEFLAGS; do \
+         case $$f in \
+           *=* | --[!k]*);; \
+           *k*) failcom='fail=yes';; \
+         esac; \
+       done; \
+       for subdir in $(TEST_SUBDIRS); do \
+         (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \
+       done; \
+       test -z "$$fail"
+
+maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests
 sc_test_names:
        @m4_builtin_rx=`echo $(m4_builtins) | sed 's/ /|/g'`; \
         m4_macro_rx="\\<($$m4_builtin_rx)\\>|\\<_?(A[CUMHS]|m4)_"; \
diff --git a/lib/Automake/tests/Makefile.am b/lib/Automake/tests/Makefile.am
index c5e53d2..fad3d7e 100644
--- a/lib/Automake/tests/Makefile.am
+++ b/lib/Automake/tests/Makefile.am
@@ -1,6 +1,7 @@
 ## Process this file with automake to create Makefile.in
 
-# Copyright (C) 2002, 2003, 2008, 2009  Free Software Foundation, Inc.
+# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation,
+# Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -28,3 +29,5 @@ Version.pl \
 Wrap.pl
 
 EXTRA_DIST = $(TESTS)
+
+include $(top_srcdir)/CheckListOfTests.am
diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index b4940db..bdf7219 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -15,7 +15,8 @@
 
 @SET_MAKE@
 
-# Copyright (C) 2002, 2003, 2008, 2009  Free Software Foundation, Inc.
+# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation,
+# Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -47,8 +48,9 @@ NORMAL_UNINSTALL = :
 PRE_UNINSTALL = :
 POST_UNINSTALL = :
 build_triplet = @build@
+DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
+       $(top_srcdir)/CheckListOfTests.am
 subdir = lib/Automake/tests
-DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in
 ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
 am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \
        $(top_srcdir)/m4/auxdir.m4 $(top_srcdir)/m4/init.m4 \
@@ -279,11 +281,14 @@ Version.pl \
 Wrap.pl
 
 EXTRA_DIST = $(TESTS)
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
 all: all-am
 
 .SUFFIXES:
 .SUFFIXES: .html .log .pl
-$(srcdir)/Makefile.in:  $(srcdir)/Makefile.am  $(am__configure_deps)
+$(srcdir)/Makefile.in:  $(srcdir)/Makefile.am 
$(top_srcdir)/CheckListOfTests.am $(am__configure_deps)
        @for dep in $?; do \
          case '$(am__configure_deps)' in \
            *$$dep*) \
@@ -518,7 +523,7 @@ maintainer-clean-generic:
        @echo "it deletes files that may require special tools to rebuild."
 clean: clean-am
 
-clean-am: clean-generic mostlyclean-am
+clean-am: clean-generic clean-local mostlyclean-am
 
 distclean: distclean-am
        -rm -f Makefile
@@ -586,17 +591,60 @@ uninstall-am:
        recheck-html
 
 .PHONY: all all-am check check-TESTS check-am check-html clean \
-       clean-generic distclean distclean-generic distdir dvi dvi-am \
-       html html-am info info-am install install-am install-data \
-       install-data-am install-dvi install-dvi-am install-exec \
-       install-exec-am install-html install-html-am install-info \
-       install-info-am install-man install-pdf install-pdf-am \
-       install-ps install-ps-am install-strip installcheck \
-       installcheck-am installdirs maintainer-clean \
+       clean-generic clean-local distclean distclean-generic distdir \
+       dvi dvi-am html html-am info info-am install install-am \
+       install-data install-data-am install-dvi install-dvi-am \
+       install-exec install-exec-am install-html install-html-am \
+       install-info install-info-am install-man install-pdf \
+       install-pdf-am install-ps install-ps-am install-strip \
+       installcheck installcheck-am installdirs maintainer-clean \
        maintainer-clean-generic mostlyclean mostlyclean-generic pdf \
        pdf-am ps ps-am recheck recheck-html uninstall uninstall-am
 
 
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+        @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+          diff='diff -u'; \
+        else \
+          diff='diff'; \
+        fi; \
+        set X $(checked_test_extensions); shift; \
+        if test $$# -eq 0; then \
+          set X $(TEST_EXTENSIONS); shift; \
+        fi; \
+        if test $$# -eq 0; then \
+          echo '$@: internal error: $$(checked_test_extensions) and' \
+               '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+          exit 255; \
+        fi; \
+        lst='$(expected_list_of_tests)'; \
+        test -n "$$lst" || lst='$(TESTS)'; \
+        for t in $$lst; do \
+          echo "$$t"; \
+        done | sort >$(am__tmk); \
+        for ext in $$*; do \
+          ls *$$ext 2>/dev/null; \
+          if test $(srcdir) != $(builddir); then \
+            (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+          fi; \
+        done | sort | uniq >$(am__tfs); \
+        if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+           result=0; \
+        else \
+           echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+           echo "+ $$diff in-makefile on-filesystem" >&2; \
+           cat $(am__tdf) >&2; \
+           result=1; \
+        fi; \
+        rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+        exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+       rm -f $(am__tmk) $(am__tfs) $(am__tdf)
+
 # Tell versions [3.59,3.63) of GNU make to not export all variables.
 # Otherwise a system limit (for SysV at least) may be exceeded.
 .NOEXPORT:
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 49d8942..e6d3098 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -847,6 +847,9 @@ $(parallel_tests)
 
 EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS)
 
+include $(top_srcdir)/CheckListOfTests.am
+maintainer-check-list-of-tests: $(parallel_tests)
+
 # Each test case depends on defs, aclocal, and automake.
 check_SCRIPTS = defs aclocal-$(APIVERSION) automake-$(APIVERSION)
 
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 78d723c..76800aa 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -50,7 +50,7 @@ POST_UNINSTALL = :
 build_triplet = @build@
 DIST_COMMON = README $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
        $(srcdir)/aclocal.in $(srcdir)/automake.in $(srcdir)/defs.in \
-       $(srcdir)/parallel-tests.am
+       $(srcdir)/parallel-tests.am $(top_srcdir)/CheckListOfTests.am
 subdir = tests
 ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
 am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \
@@ -1117,6 +1117,9 @@ yflags2.test \
 $(parallel_tests)
 
 EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS)
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
 
 # Each test case depends on defs, aclocal, and automake.
 check_SCRIPTS = defs aclocal-$(APIVERSION) automake-$(APIVERSION)
@@ -1124,7 +1127,7 @@ all: all-am
 
 .SUFFIXES:
 .SUFFIXES: .html .log .test
-$(srcdir)/Makefile.in:  $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am 
$(am__configure_deps)
+$(srcdir)/Makefile.in:  $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am 
$(top_srcdir)/CheckListOfTests.am $(am__configure_deps)
        @for dep in $?; do \
          case '$(am__configure_deps)' in \
            *$$dep*) \
@@ -1455,6 +1458,51 @@ $(parallel_tests): $(parallel_tests:-p.test=.test) 
Makefile.am
          < $(srcdir)/$$input >$@
        chmod a+rx $@
 
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+        @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+          diff='diff -u'; \
+        else \
+          diff='diff'; \
+        fi; \
+        set X $(checked_test_extensions); shift; \
+        if test $$# -eq 0; then \
+          set X $(TEST_EXTENSIONS); shift; \
+        fi; \
+        if test $$# -eq 0; then \
+          echo '$@: internal error: $$(checked_test_extensions) and' \
+               '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+          exit 255; \
+        fi; \
+        lst='$(expected_list_of_tests)'; \
+        test -n "$$lst" || lst='$(TESTS)'; \
+        for t in $$lst; do \
+          echo "$$t"; \
+        done | sort >$(am__tmk); \
+        for ext in $$*; do \
+          ls *$$ext 2>/dev/null; \
+          if test $(srcdir) != $(builddir); then \
+            (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+          fi; \
+        done | sort | uniq >$(am__tfs); \
+        if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+           result=0; \
+        else \
+           echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+           echo "+ $$diff in-makefile on-filesystem" >&2; \
+           cat $(am__tdf) >&2; \
+           result=1; \
+        fi; \
+        rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+        exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+       rm -f $(am__tmk) $(am__tfs) $(am__tdf)
+
+maintainer-check-list-of-tests: $(parallel_tests)
+
 clean-local: clean-local-check
 .PHONY: clean-local-check
 clean-local-check:
-- 
1.7.2.3


reply via email to

[Prev in Thread] Current Thread [Next in Thread]