Re: make -j not working in my documentation directory - but why?

[Ralf Wildenhues]

Hi Ed,

* Ed Hartnett wrote on Fri, Nov 27, 2009 at 12:32:20PM CET:
> We have a documentation directory, and it's the only one for which "make
> -j check" does not work. (Without the -j, it works.) 

> Here's how it fails:

> TEXINPUTS=".:$TEXINPUTS" \
>         MAKEINFO='/bin/sh /machine/netcdf/n4_new6/missing --run makeinfo
>           -I .' \
>         texi2dvi -s --pdf --batch netcdf-c.texi
> TEXINPUTS=".:$TEXINPUTS" \
>         MAKEINFO='/bin/sh /machine/netcdf/n4_new6/missing --run makeinfo
>           -I .' \
>         texi2dvi -s netcdf-c.texi
> mkdir: cannot create directory
>           `/machine/netcdf/n4_new6/man4/netcdf-install.t2d/tex_help':
>           File exists
> /usr/bin/texi2dvi: cannot create directory:
>           /machine/netcdf/n4_new6/man4/netcdf-install.t2d/tex_help

There are several reasons why parallel building of your documentation
can fail:

1)
First, there is a bug in the texi2dvi script.  It comes from texinfo.
AFAICS it has been fixed in CVS texinfo (but not yet in a released
version) with this commit:

2009-04-15  Akim Demaille  <address@hidden>

        * util/texi2dvi (ensure_dir): Die only if we could not create the
        directory at all.


2)
Then, there are more bugs/features(?) in texi2dvi that make life for you
hard: the script uses a temporary working directory that keys off of the
name of the output file, minus extension.  That means, the rules for
  netcdf-install.dvi
  netcdf-install.pdf

will try use the same temporary directory, and remove it after they're
done, leaving the other command.  Similar for the other dvi and pdf
files:

$ make -j
[...]
TEXINPUTS=".:$TEXINPUTS" \
        MAKEINFO='/bin/sh /tmp/netcdf-4.1-snapshot2009112700/missing --run 
makeinfo   -I .' \
        texi2dvi -s --pdf --batch netcdf-install.texi
TEXINPUTS=".:$TEXINPUTS" \
        MAKEINFO='/bin/sh /tmp/netcdf-4.1-snapshot2009112700/missing --run 
makeinfo   -I .' \
        texi2dvi -s --pdf --batch netcdf-c.texi
[...]
TEXINPUTS=".:$TEXINPUTS" \
        MAKEINFO='/bin/sh /tmp/netcdf-4.1-snapshot2009112700/missing --run 
makeinfo   -I .' \
        texi2dvi -s netcdf.texi
TEXINPUTS=".:$TEXINPUTS" \
        MAKEINFO='/bin/sh /tmp/netcdf-4.1-snapshot2009112700/missing --run 
makeinfo   -I .' \
        texi2dvi -s netcdf-install.texi
[...]
texi2dvi: texindex failed
make: *** [netcdf-c.dvi] Error 1
make: *** Waiting for unfinished jobs....
texi2dvi: line 868: cd: 
/tmp/netcdf-4.1-snapshot2009112700/man4/netcdf-install.t2d/dvi/bak: No such 
file or directory
make: *** [netcdf-install.dvi] Error 1
texi2dvi: etex exited with bad status, quitting.
make: *** [netcdf-tutorial.dvi] Error 1
texi2dvi: pdfetex exited with bad status, quitting.
make: *** [netcdf-c.pdf] Error 1

Luckily, texi2dvi provides a way out of this: use the --tidy flag.
You can add it to the TEXI2DVI definition in man4/Makefile.am.


3)
With some testing, I did find parallel execution failures with groff
1.18.1:

groff -man -Thtml ../f90/netcdf_f90.3 > netcdf_f90-man.html
groff -man -Thtml ../ncgen3/ncgen3.1 > ncgen3-man-1.html
groff: can't find `DESC' file
groff:fatal error: invalid device `html'
makeinfo -o netcdf.txt --no-headers ./netcdf.texi
groff: can't find `DESC' file
groff:fatal error: invalid device `html'
make: *** [netcdf_f90-man.html] Error 3
make: *** Waiting for unfinished jobs....
make: *** [ncgen3-man-1.html] Error 3

I'd still need to look into that.

If you need a workaround, you can disable parallel (GNU and some BSD)
make in this makefile completely by adding
  .NOTPARALLEL:

to man4/Makefile.am.

Cheers,
Ralf