parallel autotest revisited

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm trying to get parallel tests working on cygwin, and the use of <> on a
fifo is a sticking point.  With this patch, things still pass on Linux and
Solaris, so I believe it is more portable.  The idea is that rather than
relying on the non-POSIX read-write fifo, we instead open a write fifo in
each test group and a read fifo in the master process.  POSIX requires
opening a unidirectional fifo to block until someone else opens the
counterpart direction, so we must delay the read fd until after we have
spawned at least one background task that is blocked on opening a write fd.

It doesn't reliably fix things for cygwin 1.5 (which has a bug where
opening a fifo for writing doesn't block when there is no current reader,
such that subsequent writes can inappropriately fail cause readers to hang
waiting for data), but whereas all parallel tests failed without the
patch, I actually got a mix of some successful runs instead of hangs with
this patch.  Meanwhile, the unreleased cygwin 1.7 currently has a
regression in named pipe handling which prevents either pre-patch or
post-patch from working, but the cygwin developers are aware of it[1] and
hope to have a fix for the non-bidirectional handling before cygwin 1.7 is
released (which would thus make parallel testsuites work on cygwin 1.7
with this patch), but with no promise for bidirectional pipes[2] (meaning
that parallel testsuites are still a problem for cygwin if this patch is
not applied).

[1] http://cygwin.com/ml/cygwin/2009-07/msg00258.html
[2] http://cygwin.com/ml/cygwin/2009-07/msg00093.html

Does this patch look correct?  Should I go ahead and apply it?

To test it, you can use:
$ git pull git://repo.or.cz/autoconf/ericb.git parallel

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpTTAQACgkQ84KuGfSFAYAlvACeNTNxuT+gUXJ/B/Pr9viEFMnI
Ya4AoKUfNaa1fhkdt7RWpANmy9e5oO7g
=gqo+
-----END PGP SIGNATURE-----

>From df9a62519ad46fbd2ee72575ca480a7cb7983ae3 Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Tue, 7 Jul 2009 06:58:18 -0600
Subject: [PATCH] Make parallel testsuite more portable.

* lib/autotest/general.m4 (AT_INIT) <AT_JOB_FIFO_FD>: Avoid <>;
instead open write descriptor in each group and read descriptor in
main driver.
* tests/autotest.at (AT_SKIP_PARALLEL_TESTS): Relax condition.

Signed-off-by: Eric Blake <address@hidden>
---
 ChangeLog               |    8 ++++++++
 lib/autotest/general.m4 |   10 ++++++----
 tests/autotest.at       |    3 +--
 3 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 68c4490..ff8e9ce 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2009-07-07  Eric Blake  <address@hidden>
+
+       Make parallel testsuite more portable.
+       * lib/autotest/general.m4 (AT_INIT) <AT_JOB_FIFO_FD>: Avoid <>;
+       instead open write descriptor in each group and read descriptor in
+       main driver.
+       * tests/autotest.at (AT_SKIP_PARALLEL_TESTS): Relax condition.
+
 2009-07-03  Eric Blake  <address@hidden>

        Avoid syntax error in ash.
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 50c2ff7..63b0525 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -1296,9 +1296,7 @@ at_first=:
 if test $at_jobs -ne 1 &&
      rm -f "$at_job_fifo" &&
      test -n "$at_job_group" &&
-     ( mkfifo "$at_job_fifo" && eval 'exec AT_JOB_FIFO_FD<> "$at_job_fifo"' \
-       && trap 'exit 1' PIPE STOP TSTP ) 2>/dev/null &&
-     eval 'exec AT_JOB_FIFO_FD<> "$at_job_fifo"'
+     ( mkfifo "$at_job_fifo" && trap 'exit 1' PIPE STOP TSTP ) 2>/dev/null
 then
   # FIFO job dispatcher.

@@ -1344,6 +1342,7 @@ dnl avoid all the status output by the shell.
     (
       # Start one test group.
       $at_job_control_off
+      exec AT_JOB_FIFO_FD>"$at_job_fifo"
 dnl When a child receives PIPE, be sure to write back the token,
 dnl so the master does not hang waiting for it.
 dnl errexit and xtrace should not be set in this shell instance,
@@ -1377,7 +1376,10 @@ dnl          kill -13 $$
       set x $[*]
     fi
     test -f "$at_stop_file" && break
-    at_first=false
+    if $at_first; then
+      at_first=false
+      exec AT_JOB_FIFO_FD<"$at_job_fifo"
+    fi
   done
   # Read back the remaining ($at_jobs - 1) tokens.
   set X $at_joblist
diff --git a/tests/autotest.at b/tests/autotest.at
index e83b329..b4ba887 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -992,9 +992,8 @@ m4_define([AT_SKIP_PARALLEL_TESTS],
 # limited conditions; help is appreciated in widening this test base.
 AT_CHECK([${CONFIG_SHELL-$SHELL} -c 'test -n "${BASH_VERSION+set}]]dnl
 [[${ZSH_VERSION+set}${TEST_PARALLEL_AUTOTEST+set}"' || exit 77])
-# The parallel scheduler requires mkfifo and bidirectional redirection to work.
+# The parallel scheduler requires mkfifo to work.
 AT_CHECK([mkfifo fifo || exit 77])
-AT_CHECK([${CONFIG_SHELL-$SHELL} -c 'eval "exec 5<>fifo"' || exit 77])
 ])


-- 
1.6.3.3.334.g916e1