bug#13832: Tests fail because make runs with more than one jobs

[Joshua C.]

2013/3/6 Joshua C. <address@hidden>

2013/3/5 Stefano Lattarini <address@hidden>:

> tags 13832 + moreinfo
> severity 13832 minor
> stop
>
> Hi Joshua, sorry for the delay.

Don't worry, I'm glad that I got a response, because sometimes some maintainers (on other projects) don't even bother to respond but this is another story...

>
> On 02/27/2013 10:07 AM, Joshua C. wrote:
>> In the last couple of days I encountered a number of different tests
>> that reported a failure with ‘make check’. However in order to save
>> some time I ran ‘make –j4 check’ and this saved me about 75% of the
>> check time (~16min compared to ~62min for ‘make –j1 check’). The
>> reportedly failed tests passed without errors when tested with ‘make
>> –j1 check’. This makes me believe that ‘make’ doesn’t like to be run
>> as a server.
>>
> That is weird, I routinely run the tests with high parallelism myself
> (even with "make -j16"), and have never encountered such an issue.
>
> Can you please send us more details about your system, as well as the
> content of the file 'test-suite.log' left behind after the testsuite
> run?
>

This is a fedora-17 (gcc-4.7.2) with updated autoconf, automake, make, flex and bison. I've uploaded the 'test-suite.log' file and see that it has some more information about the  installed packages.

>> My proposition is: if a test fails for some reason,
>>
> They are not expected to fail.  So we should work out the underlying
> reason for the problem, and fix it.
>
>> can we automatically re-run it with passing ‘-j1’ to make?
>>
>> If it still fails then count the result as a failure, if not – as a
>> success. Can someone take a look at it tries to implement it? I’m
>> not that good at experimenting with make-targets.
>>
>> Tested with:
>>
>> automake: git a5ed87e7944deaea33914230e3d67ff08eb18382
>> autoconf: git 9e087dd343b67ac58a7f54e08335ac62fc33ed53
>> make: git 4baf9ab4564447355b5748d1375959e817771d17
>>
>> Failing tests with ‘make –j4 check’ (these pass fine with ‘make –j1 check’)
>>
>> t/objc-megademo.sh
>> t/ar-lib4.sh
>>
>>
>> --joshua
>>
>>
>
> Regards,
>   Stefano
>

--
--joshua

I made some tests and came up with the following patch. Now the failing tests pass without errors. The patch is quite trivial; hopefully some can come up with a better solution.

>From 85f520da2cddc60db7d43efa424ed4d9ca1aef50 Fri Mar 15 01:53:51 2013
From: Joshua Covington <address@hidden>
Date: Fri, 15 Mar 2013 01:53:51 +0100
Subject: [PATCH] fix broken tests

---
 t/ar-lib4.sh       | 3 ++-
 t/objc-megademo.sh | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/t/ar-lib4.sh b/t/ar-lib4.sh
index 68615e8..534f466 100755
--- a/t/ar-lib4.sh
+++ b/t/ar-lib4.sh
@@ -35,7 +35,8 @@ END
 
 libtoolize
 $ACLOCAL
-AUTOMAKE_fails
+sleep 1
+AUTOMAKE_fails --add-missing
 
 grep 'requires.*AM_PROG_AR' stderr
 
diff --git a/t/objc-megademo.sh b/t/objc-megademo.sh
index 07764cd..8a72811 100755
--- a/t/objc-megademo.sh
+++ b/t/objc-megademo.sh
@@ -80,6 +80,7 @@ play_LDADD = libfoo.la
 play_LDFLAGS = -lobjc
 lib_LTLIBRARIES = libfoo.la
 libfoo_la_SOURCES = foo.h foo.c fooxx.cxx fooo.m foooxx.mm
address@hidden@ --tag=CC
 END
 
 ## Run Autotools.
-- 
1.7.11.7

-- 
--joshua

0003-fix-broken-tests.patch
Description: Binary data