Re: Help2man and Parallel Make Race

[Stefano Lattarini]

On 11/19/2012 08:55 PM, Neil T. Dantam wrote:
> 
> Hi,
> 
> I'm using help2man to build manpages that I'd like to distribute with
> the source tarball.  I followed the instructions/example in the
> automake manual, which has manpage foo.1 depend on foo.c and
> configure.ac, then does a recursive make to build foo$(EXEEXT):
> <http://www.gnu.org/software/automake/manual/html_node/Errors-with-distclean.html>
> 
> This does solve the problem of unnecessarily rebuilding manpages when
> the binary is rebuilt/relinked, but it also seems to introduce a race
> condition during parallel makes.  The problem is that there are now
> two ways to build foo$(EXEEXT): (1) the normal automake rule from
> bin_PROGRAMS/foo_SOURCES and (2) the recursive make for the rule for
> foo.1.
>
> To give a more concrete example ...
>
> [SNIP]
>
Yes, and I seem to recall that a similar problem occurred already in an
important real-world package (GNU coreutils maybe?).

> So, it looks like the second `mv -f .deps/achd.Tpo .deps/achd.Po' is
> failing because the dependency file was already moved.  Is there
> something silly that I've missed here?
>
Nope, you've found a real race-condition.

> It seems that possible resolutions are:
> * Don't do parallel makes
>
Yikes!  Unacceptable.

> * Don't distribute prebuilt manpages
>
That's the route that GNU coreutils has taken recently; in the unlikely
case perl is absent on the the build system, dummy man-pages are
generated.

> * Build the manpages in a SUBDIR that gets built after .
>
This won't save packages that want to avoid recursive make, sadly.

Assuming GNU make, a good solution would be to use order-only
dependencies:

   foo.1: foo.c | foo$(EXEEXT)
           help2man ...

The above ensures that foo$(EXEEXT) is built before foo.1, but doesn't
cause foo.1 to be rebuilt just because foo$(EXEEXT) has changed.  Which
is exactly what we want.

Needless to say, this solution is highly unportable to non-GNU makes,
sigh.  Still, this fix has a place in the Automake-NG manual (since
there GNU make is assumed anyway).  I'll write a patch in the next
days.

> Using SUBDIRs seems like the best of those options, though still not
> ideal since make may often recurse unnecessarily.  Is there any way to
> do this without more recursive makes?
>
Good question.  The recent issues with the GNU coreutils build system
seem to suggest the answer is "no, not really".  I'd love if someone
could prove me wrong.  Otherwise, we'll just have to amend the manual
to reference the possibility of a race, and explain that no real good
solution exists that doesn't involve either extra recursion or some
unportable GNU make feature.

Regards,
  Stefano