Re: parallel NetBSD make quirks

[Ralf Wildenhues]

[ adding automake-patches ]

* Ralf Wildenhues wrote on Sun, May 17, 2009 at 01:56:39PM CEST:
> NetBSD (and other BSD) `make -jN' without `-B'
> 
> - reuses shells between multiple commands in one rule
>   (without resetting the working directory, environment variables etc),
> 
> - sprinkles "--- $target ---" status messages through the output,

(HP-UX 'make -P' does something similar)

> - does not route standard error of sub processes (sub makes as well as
>   rule commands) correctly: all appears on stdout, except the stderr of
>   the outermost `make' process.
[...]

I forgot to mention that it also mangles leading white space in the
output.

> Patch for Autoconf manual coming up.
> 
> Note that the Debian freebsd-make -jN has more issues which I could
> however not reproduce on a real NetBSD; guess that's due to Debian
> packaging an older version of the tool.

Here's that patch.  OK to commit?

BTW, BSD make authors really should take a few hints from the GNU make
implementation.  Despite their "optimization" of reusing shells, their
parallel make surely still feels quite a bit slower than GNU make's.
I don't have numbers to prove this, but I can measure if anyone is
interested.

Thanks,
Ralf

2009-05-17  Ralf Wildenhues  <address@hidden>

        New manual section `Parallel Make'.
        * doc/autoconf.texi (Parallel Make): New node, document NetBSD
        `make -jN' quirks.
        (Top, Portable Make): Adjust menus.

diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 928b417..2e5a645 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -509,6 +509,7 @@ Top
 * Macros and Submakes::         @code{make macro=value} and submakes
 * The Make Macro MAKEFLAGS::    @code{$(MAKEFLAGS)} portability issues
 * The Make Macro SHELL::        @code{$(SHELL)} portability issues
+* Parallel Make::               Parallel @command{make} quirks
 * Comments in Make Rules::      Other problems with Make comments
 * obj/ and Make::               Don't name a subdirectory @file{obj}
 * make -k Status::              Exit status of @samp{make -k}
@@ -17567,6 +17568,7 @@ Portable Make
 * Macros and Submakes::         @code{make macro=value} and submakes
 * The Make Macro MAKEFLAGS::    @code{$(MAKEFLAGS)} portability issues
 * The Make Macro SHELL::        @code{$(SHELL)} portability issues
+* Parallel Make::               Parallel @command{make} quirks
 * Comments in Make Rules::      Other problems with Make comments
 * obj/ and Make::               Don't name a subdirectory @file{obj}
 * make -k Status::              Exit status of @samp{make -k}
@@ -17878,6 +17880,66 @@ The Make Macro SHELL
 sh
 @end example
 
address@hidden Parallel Make
address@hidden Parallel Make
address@hidden Parallel @command{make}
+
+Support for parallel execution in @command{make} implementation varies.
+Generally, using @acronym{GNU} make is your best bet.  When NetBSD
address@hidden is invoked with @address@hidden, it will reuse the
+same shell for multiple commands within one recipe.  This can have
+unexpected address@hidden that @acronym{GNU} make has
+heuristics to avoid spawning a shell at all if the command is deemed
+safe to be executed directly.} For example, change of directories or
+variables persist between commands:
+
address@hidden
+all:
+        @@var=value; cd /; pwd; echo $$var; echo $$$$
+        @@pwd; echo $$var; echo $$$$
address@hidden example
+
address@hidden
+may output the following with @code{make -j1}:
+
address@hidden
+--- all ---
+/
+value
+32235
+/
+value
+32235
address@hidden example
+
+while without @option{-j1}, or with @option{-B}, the output looks less
+surprising:
+
address@hidden
+/
+value
+32238
+/tmp
+
+32239
address@hidden example
+
+Another consequence of this is that, if one command in a recipe uses
address@hidden 0} to indicate a successful exit, the shell will be gone
+and the remaining commands of this recipe will not be executed.
+
+The above example also shows additional status output NetBSD
address@hidden produces in parallel mode for targets being updated.
+
+Furthermore, parallel NetBSD @command{make} will route standard error
+output from commands that it spawns into its own standard output, and
+may remove leading whitespace from output lines.
+
+You can avoid these issues by using the @option{-B} option to enable
+compatibility semantics.  However, that will effectively also disable
+all parallelism as that will cause prerequisites to be updated in the
+order they are listed in a rule.
+
 @node Comments in Make Rules
 @section Comments in Make Rules
 @cindex Comments in @file{Makefile} rules