[Ping PATCHES] {master} Optimize tests `instspc-*.test' for speed.

[Stefano Lattarini]

Hello automakers. 

References:
  <http://lists.gnu.org/archive/html/automake-patches/2010-11/msg00152.html>
  <http://lists.gnu.org/archive/html/automake-patches/2010-12/msg00006.html>

Now that the spurious failures of `instspc-*.test' on Tru64 have been fixed
(kudos to Ralf for doing that), I can finally ping this couple of patches.

In the meantime, I've rebased them on master, and improved (and simplified)
the first patch by taking advantage of the "inter-tests dependency" feature
of the automake parallel testsuite driver.

The updated patches are attached.

OK for master?

Regards,
   Stefano

From dc36d49a4b629c592a3d66f63bd9a1ebeba89084 Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Fri, 5 Nov 2010 15:51:56 +0100
Subject: [PATCH 1/2] tests: optimize `instspc-*.test' for speed

After the split of `instspc.test' into various generated tests,
the running time of the testsuite has noticeably increased, since
all these new generated tests must run aclocal, autoconf and
automake, whereas previously they were run only once (at the
beginning of `instspc.test').  But luckily, since the new tests
share the same input files for the autotools, this situation can
be easily worked around (at the expenses of a slight increase of
complexity for the testsuite scaffolding).

* tests/instspc-data.test: New helper test, properly calling
the `instspc-tests.sh' script to generate input data for the
others `instspc-*.test' tests.
* tests/Makefile.am (TESTS): Add `instspc-data.test'.
($(instspc_tests:.test=.log)): Depend on its log file.
(instspc-data.log): Depend on `instspc-tests.sh'.
* tests/instspc-tests.sh: Recognize new action `generate-data',
and use it to create hand-written and autotools-generated static
files shared by all the `instspc-*.test' tests.
When sourced by the `instspc-*.test' tests, use those previously
created files instead of recreating them from scratch.
(deindent, create_input_data): New subroutines.
Some other related changes and refactorings.

From a suggestion by Ralf Wildenhues.
---
 ChangeLog               |   26 +++++
 tests/Makefile.am       |    8 +-
 tests/Makefile.in       |    8 +-
 tests/instspc-data.test |   26 +++++
 tests/instspc-tests.sh  |  233 +++++++++++++++++++++++++++-------------------
 5 files changed, 201 insertions(+), 100 deletions(-)
 create mode 100755 tests/instspc-data.test

diff --git a/ChangeLog b/ChangeLog
index 18567c6..9aa80a1 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,29 @@
+2011-01-25  Stefano Lattarini  <address@hidden>
+
+       tests: optimize `instspc-*.test' for speed
+       After the split of `instspc.test' into various generated tests,
+       the running time of the testsuite has noticeably increased, since
+       all these new generated tests must run aclocal, autoconf and
+       automake, whereas previously they were run only once (at the
+       beginning of `instspc.test').  But luckily, since the new tests
+       share the same input files for the autotools, this situation can
+       be easily worked around (at the expenses of a slight increase of
+       complexity for the testsuite scaffolding).
+       * tests/instspc-data.test: New helper test, properly calling
+       the `instspc-tests.sh' script to generate input data for the
+       others `instspc-*.test' tests.
+       * tests/Makefile.am (TESTS): Add `instspc-data.test'.
+       ($(instspc_tests:.test=.log)): Depend on its log file.
+       (instspc-data.log): Depend on `instspc-tests.sh'.
+       * tests/instspc-tests.sh: Recognize new action `generate-data',
+       and use it to create hand-written and autotools-generated static
+       files shared by all the `instspc-*.test' tests.
+       When sourced by the `instspc-*.test' tests, use those previously
+       created files instead of recreating them from scratch.
+       (deindent, create_input_data): New subroutines.
+       Some other related changes and refactorings.
+       From a suggestion by Ralf Wildenhues.
+
 2011-01-23  Stefano Lattarini  <address@hidden>
 
        coverage: test semantics of "dummy" per-target flags
diff --git a/tests/Makefile.am b/tests/Makefile.am
index f53a1f3..a4cb5b4 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -85,8 +85,11 @@ $(instspc_tests): Makefile.am
          } > address@hidden
        $(AM_V_at)chmod a+rx address@hidden && mv -f address@hidden $@
 
-# All instspc*.test tests work by sourcing the instspc-tests.sh script.
-$(instspc_tests:.test=.log): instspc-tests.sh
+# All instspc-*.test tests work by sourcing the `instspc-tests.sh'
+# script.  Also, they all use shared data generated by the helper
+# test `instspc-data.test', for reasons of speed.
+instspc-data.log: instspc-tests.sh
+$(instspc_tests:.test=.log): instspc-tests.sh instspc-data.log
 
 MAINTAINERCLEANFILES += $(instspc_tests)
 EXTRA_DIST += instspc-tests.sh
@@ -504,6 +507,7 @@ instman2.test \
 instmany.test \
 instmany-mans.test \
 instmany-python.test \
+instspc-data.test \
 $(instspc_tests) \
 interp.test \
 interp2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index b78e5a5..5b9e7f1 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -767,6 +767,7 @@ instman2.test \
 instmany.test \
 instmany-mans.test \
 instmany-python.test \
+instspc-data.test \
 $(instspc_tests) \
 interp.test \
 interp2.test \
@@ -1652,8 +1653,11 @@ $(instspc_tests): Makefile.am
          } > address@hidden
        $(AM_V_at)chmod a+rx address@hidden && mv -f address@hidden $@
 
-# All instspc*.test tests work by sourcing the instspc-tests.sh script.
-$(instspc_tests:.test=.log): instspc-tests.sh
+# All instspc-*.test tests work by sourcing the `instspc-tests.sh'
+# script.  Also, they all use shared data generated by the helper
+# test `instspc-data.test', for reasons of speed.
+instspc-data.log: instspc-tests.sh
+$(instspc_tests:.test=.log): instspc-tests.sh instspc-data.log
 
 # Each test case depends on defs, aclocal, and automake.
 $(TEST_LOGS): defs aclocal-$(APIVERSION) automake-$(APIVERSION)
diff --git a/tests/instspc-data.test b/tests/instspc-data.test
new file mode 100755
index 0000000..1c165d8
--- /dev/null
+++ b/tests/instspc-data.test
@@ -0,0 +1,26 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Helper testcase which generate input data for the other test
+# `instspc-*.test'.  It basically delegates the work to the helper
+# script `instspc-test.sh'.
+
+# Ensure proper definition of $testsrcdir.
+. ./defs-static || exit 99
+test -n "$testsrcdir" || exit 99 # sanity check
+
+instspc_action=generate-data
+. $testsrcdir/instspc-tests.sh
diff --git a/tests/instspc-tests.sh b/tests/instspc-tests.sh
index 1be8d0e..6add74f 100755
--- a/tests/instspc-tests.sh
+++ b/tests/instspc-tests.sh
@@ -21,18 +21,21 @@
 # Original report from James Amundson about file names with spaces.
 # Other characters added by Paul Eggert.
 #
-# This script fulfills a double role:
+# This script fulfills a threefold role:
 #   1. It generates a Makefile.am snippet, containing the definition
 #      of proper lists of tests.
-#   2. It is sourced by said generated tests with proper parameters
+#   2. It sets up a directory containing some common data files and
+#      autotools-generated files used by said generated tests (this
+#      is done for speed reasons only).
+#   3. It is sourced by said generated tests with proper parameters
 #      pre-set, to run the "meat" of the checks.
-# This setup might seem tricky and over-engineered abuse, but past
+# This setup might seem a tricky and over-engineered abuse, but past
 # (painful) experiences showed that it is indeed required, because
 # the test generation code and test execution code tend to be
 # inextricably coupled and intertwined.
 #
 
-# Be more Bourne compatible (snippet copied from `tests/defs.in').
+# Be more Bourne compatible (snippet copied from `tests/defs').
 DUALCASE=1; export DUALCASE # for MKS sh
 if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then
   emulate sh
@@ -58,21 +61,23 @@ if test x"$instspc_action" = x; then
 elif test $# -gt 0; then
   echo "$0: action specified and command line arguments used" >&2
   exit 99
-elif test x"$instspc_action" = x"generate-makefile"; then
-  :
-else
-  case $instspc_action in
-    test-build|test-install)
-      if test x"$instspc_test_name" = x; then
-        echo "$0: test name undefined for action '$instspc_action'" >&2
-        exit 99
-      fi;;
-    *)
-      echo "$0: invalid action: '$instspc_action'"
-      exit 99;;
-  esac
 fi
 
+case $instspc_action in
+  generate-makefile|generate-data)
+    ;;
+  test-build|test-install)
+    if test x"$instspc_test_name" = x; then
+      echo "$0: test name undefined for action '$instspc_action'" >&2
+      exit 99
+    fi
+    ;;
+  *)
+    echo "$0: invalid action: '$instspc_action'"
+    exit 99
+    ;;
+esac
+
 # Helper subroutine for test data definition.
 # Usage: define_problematic_string NAME STRING
 define_problematic_string ()
@@ -94,6 +99,91 @@ define_problematic_string ()
   esac
 }
 
+# Helper subroutines for creation of input data files.
+
+deindent ()
+{
+  sed 's/^ *//' # we don't strip leading tabs -- this is deliberate!
+}
+
+create_input_data ()
+{
+  mkdir sub
+
+  deindent > configure.in << 'EOF'
+    AC_INIT([instspc], [1.0])
+    AM_INIT_AUTOMAKE
+    AC_CONFIG_FILES([Makefile])
+    AC_PROG_CC
+    AC_PROG_RANLIB
+    AC_OUTPUT
+EOF
+
+  : > sub/base.h
+  : > sub/nobase.h
+  : > sub/base.dat
+  : > sub/nobase.dat
+  : > sub/base.sh
+  : > sub/nobase.sh
+
+  deindent > source.c << 'EOF'
+    int
+    main (int argc, char **argv)
+    {
+      return 0;
+    }
+EOF
+  cp source.c source2.c
+
+  deindent > Makefile.am << 'EOF'
+    foodir = $(prefix)/foo
+    fooexecdir = $(prefix)/foo
+
+    foo_HEADERS = sub/base.h
+    nobase_foo_HEADERS = sub/nobase.h
+
+    dist_foo_DATA = sub/base.dat
+    nobase_dist_foo_DATA = sub/nobase.dat
+
+    dist_fooexec_SCRIPTS = sub/base.sh
+    nobase_dist_fooexec_SCRIPTS = sub/nobase.sh
+
+    fooexec_PROGRAMS = sub/base
+    nobase_fooexec_PROGRAMS = sub/nobase
+    sub_base_SOURCES = source.c
+    sub_nobase_SOURCES = source.c
+
+    fooexec_LIBRARIES = sub/libbase.a
+    nobase_fooexec_LIBRARIES = sub/libnobase.a
+    sub_libbase_a_SOURCES = source.c
+    sub_libnobase_a_SOURCES = source.c
+
+    .PHONY: test-install-sep
+    test-install-sep: install
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.h'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.h'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.h'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.dat'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.dat'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.dat'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.sh'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.sh'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.sh'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase$(EXEEXT)'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase$(EXEEXT)'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base$(EXEEXT)'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/libnobase.a'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/libnobase.a'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/libbase.a'
+EOF
+
+  $ACLOCAL
+  $AUTOCONF
+  $AUTOMAKE -a
+
+  : > success
+}
+
 # Be sure to avoid interferences from the environment.
 instspc_names_list=''
 instspc_xfail_builds_list=''
@@ -189,102 +279,54 @@ if test x"$instspc_action" = x"generate-makefile"; then
   exit 0
 fi
 
-###  If we are still here, we have to run a test ...
-
-# We'll need the full setup provided by `tests/defs'.  Temporarly disable
+# We'll need the full setup provided by `tests/defs'.  Temporarily disable
 # the errexit flag, since the setup code might not be prepared to deal
 # with it.
 set +e
 . ./defs || Exit 99
 set -e
 
+# The directory set up by the `generate-data' action should contain all
+# the files we need.  So remove the other files created by ./defs.  And
+# check we really are in a temporary `*.dir' directory in the build tree,
+# since the last thing we want is to remove some random user files!
+test -f ../defs-static && test -f ../defs || Exit 99
+case `pwd` in *.dir);; *) Exit 99;; esac
+rm -f *
+
+if test x"$instspc_action" = x"generate-data"; then
+  # We must *not* remove the test directory, since its contents must be
+  # used by following dependent tests.
+  trap 'Exit $?' 0
+  create_input_data
+  Exit 0
+fi
+
+###  If we are still here, we have to run a test ...
+
 eval "instspc_test_string=\${instspc__$instspc_test_name}" || Exit 99
 if test x"$instspc_test_string" = x; then
   echo "$me: invalid test name: '$instspc_test_name'" >&2
   Exit 99
 fi
 
-# Skip if this system doesn't support these characters in file names.
-mkdir "./$instspc_test_string" || Exit 77
-
-mkdir sub sub1
-
-cat >> configure.in << 'EOF'
-AC_PROG_CC
-AC_PROG_RANLIB
-AC_OUTPUT
-EOF
-
-: > sub/base.h
-: > sub/nobase.h
-: > sub/base.dat
-: > sub/nobase.dat
-: > sub/base.sh
-: > sub/nobase.sh
-
-cat > source.c << 'EOF'
-int
-main (int argc, char **argv)
-{
-  return 0;
+test -f ../instspc-data.dir/success || {
+  echo "$me: setup by instspc-data.test failed" >&2
+  Exit 99
 }
-EOF
-cp source.c source2.c
-
-cat > Makefile.am << 'EOF'
-foodir = $(prefix)/foo
-fooexecdir = $(prefix)/foo
-
-foo_HEADERS = sub/base.h
-nobase_foo_HEADERS = sub/nobase.h
-
-dist_foo_DATA = sub/base.dat
-nobase_dist_foo_DATA = sub/nobase.dat
-
-dist_fooexec_SCRIPTS = sub/base.sh
-nobase_dist_fooexec_SCRIPTS = sub/nobase.sh
-
-fooexec_PROGRAMS = sub/base
-nobase_fooexec_PROGRAMS = sub/nobase
-sub_base_SOURCES = source.c
-sub_nobase_SOURCES = source.c
 
-fooexec_LIBRARIES = sub/libbase.a
-nobase_fooexec_LIBRARIES = sub/libnobase.a
-sub_libbase_a_SOURCES = source.c
-sub_libnobase_a_SOURCES = source.c
-
-.PHONY: test-install-sep
-test-install-sep: install
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.h'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.h'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.h'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.dat'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.dat'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.dat'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.sh'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.sh'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.sh'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase$(EXEEXT)'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase$(EXEEXT)'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base$(EXEEXT)'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/libnobase.a'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/libnobase.a'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/libbase.a'
-EOF
-
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE -a
+# Skip if this system doesn't support these characters in file names.
+mkdir "./$instspc_test_string" || Exit 77
 
 case $instspc_action in
   test-build)
-    build=$instspc_test_string
-    dest=`pwd`/sub1
+    dest=`pwd`/_dest
+    relbuilddir=../..
+    cd "./$instspc_test_string"
     ;;
   test-install)
-    build=sub1
     dest=`pwd`/$instspc_test_string
+    relbuilddir=..
     ;;
   *)
     echo "$me: internal error: invalid action '$instspc_action'"
@@ -292,9 +334,8 @@ case $instspc_action in
     ;;
 esac
 
-cd "./$build"
-
-../configure --prefix "/$instspc_test_string-prefix"
+$relbuilddir/instspc-data.dir/configure \
+  --prefix "/$instspc_test_string-prefix"
 $MAKE
 # Some make implementations eliminate leading and trailing whitespace
 # from macros passed on the command line, and some eliminate leading
-- 
1.7.2.3

From 998400ce28ea1ff9b956a80fb23f88fb6e8f41d6 Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Sun, 14 Nov 2010 18:10:37 +0100
Subject: [PATCH 2/2] tests: `instspc-*.test': do not create useless source file

* tests/instspc-tests.sh (create_input_data): Do not create
unused source file `source2.c'.
---
 ChangeLog              |    6 ++++++
 tests/instspc-tests.sh |    1 -
 2 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 9aa80a1..828e893 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
 2011-01-25  Stefano Lattarini  <address@hidden>
 
+       tests: `instspc-*.test': do not create useless source file
+       * tests/instspc-tests.sh (create_input_data): Do not create
+       unused source file `source2.c'.
+
+2011-01-25  Stefano Lattarini  <address@hidden>
+
        tests: optimize `instspc-*.test' for speed
        After the split of `instspc.test' into various generated tests,
        the running time of the testsuite has noticeably increased, since
diff --git a/tests/instspc-tests.sh b/tests/instspc-tests.sh
index 6add74f..b78c9eb 100755
--- a/tests/instspc-tests.sh
+++ b/tests/instspc-tests.sh
@@ -133,7 +133,6 @@ EOF
       return 0;
     }
 EOF
-  cp source.c source2.c
 
   deindent > Makefile.am << 'EOF'
     foodir = $(prefix)/foo
-- 
1.7.2.3