coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[PATCH] tests: avoid gross inefficiency in "make test"


From: Jim Meyering
Subject: [PATCH] tests: avoid gross inefficiency in "make test"
Date: Wed, 09 Feb 2011 17:26:36 +0100

Running "make -j25 check" on a nominal-12-core F14 system would
cause serious difficulty leading to an OOM kill -- and this is brand new.
It worked fine yesterday.  I tracked it down to all of the make processes
working on the "built_programs.list" (in src/Makefile.am) rule

built_programs.list:
        @echo $(bin_PROGRAMS) $(bin_SCRIPTS) | tr ' ' '\n' \
          | sed -e 's,$(EXEEXT)$$,,' | $(ASSORT) -u | tr '\n' ' '

Which made me realize we were running that submake over 400 times,
once per test scripts (including skipped ones).  That's well worth
avoiding, even if it means a new temporary file.

I don't know the root cause of the OOM-kill (preceded by interminable
minutes of a seemingly hung and barely responsive system) or why it started
happening today (afaics, none of the programs involved was updated),
but this does fix it...

>From 8af39566de60b106146eea757f2611352f39544f Mon Sep 17 00:00:00 2001
From: Jim Meyering <address@hidden>
Date: Wed, 9 Feb 2011 17:08:58 +0100
Subject: [PATCH] tests: avoid gross inefficiency in "make test"

Do not run a sub-make to set up the environment for each
and every test script.  Instead, run it just once and store
the result in a file.
* tests/check.mk (built_programs): Remove definition.
(.built-programs): New rule to create the temporary file.
(CLEANFILES): Arrange to remove it.
(TESTS_ENVIRONMENT): Simply cat .built-programs, rather than
running the sub-make.
* .gitignore: Ignore it.
---
 .gitignore     |    1 +
 tests/check.mk |   11 ++++++++---
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/.gitignore b/.gitignore
index 7fead3d..138e72a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@
 /stamp-h1
 /tests/*/*.log
 /tests/t?
+/tests/.built-programs
 /tests/test-suite.log
 ID
 Makefile
diff --git a/tests/check.mk b/tests/check.mk
index f931806..b80dce8 100644
--- a/tests/check.mk
+++ b/tests/check.mk
@@ -40,8 +40,13 @@ vc_exe_in_TESTS: Makefile
 check: vc_exe_in_TESTS
 .PHONY: vc_exe_in_TESTS

-built_programs = \
-  (cd $(top_builddir)/src && MAKEFLAGS= $(MAKE) -s built_programs.list)
+CLEANFILES =
+CLEANFILES += .built-programs
+check-am: .built-programs
+.built-programs:
+       $(AM_V_GEN)cd $(top_builddir)/src                               \
+            && MAKEFLAGS= $(MAKE) -s built_programs.list               \
+          > address@hidden && mv address@hidden $@

 # Note that the first lines are statements.  They ensure that environment
 # variables that can perturb tests are unset or set to expected values.
@@ -76,7 +81,7 @@ TESTS_ENVIRONMENT =                           \
   abs_top_builddir='$(abs_top_builddir)'       \
   abs_top_srcdir='$(abs_top_srcdir)'           \
   abs_srcdir='$(abs_srcdir)'                   \
-  built_programs="`$(built_programs)`"         \
+  built_programs="`cat .built-programs`"       \
   host_os=$(host_os)                           \
   host_triplet='$(host_triplet)'               \
   srcdir='$(srcdir)'                           \
--
1.7.4.2.g597a6



reply via email to

[Prev in Thread] Current Thread [Next in Thread]