Re: make calling script calling make results in read jobs pipe: Bad file descriptor. Stop.

[Paul Smith]

On Thu, 2021-03-04 at 12:24 -0500, Martin d'Anjou wrote:
> The process tree is like this:
> gnumake(1) -> cmdA(2) -> cmdB(3) -> cmdC(4)-> ... -> gnumake(7)
> 
> The first make is called with the -j option. The intermediate
> commands are not controlled by me, they are proprietary vendor code
> that I cannot access. The last call to make is also hidden from me.

> Help me understand what is happening. Is it a limitation/bug of GNU
> Make, an unsupportable use case, what is happening? Any suggestions
> on how to fix this?

There are a number of recent issues filed around this issue.

What happens is that the top-level make is started with -jN so it
creates the jobserver infrastructure, which is a pipe.  make also puts
the file descriptors of the pipe into the MAKEFLAGS variable so that
sub-makes can find it.

When make invokes a command in a recipe, if it thinks that the command
is a sub-make it will leave the pipes open so that the sub-make can
participate in the jobserver.

If the top-level make believes that the command is NOT a sub-make, it
closes the pipes so that the sub-command won't get confused about these
extra open file descriptors.

I assume that in your situation, make is determining that cmdA is not a
sub-make and so is closing the pipes.

However, make ALWAYS sends along the MAKEFLAGS variable to all commands
it runs, whether they are sub-makes or not.  And, there is a bug (filed
in Savannah) that even when make determines that the process is not a
sub-make, it will still send along the jobserver information in
MAKEFLAGS.

This normally is not a problem.  Process that are not make obviously
don't care about MAKEFLAGS.  If the sub-process IS a make and sees
MAKEFLAGS it will try to use the jobserver pipes, see they are closed,
and notify the user that jobserver is not available.

However, there are problems in situations like this:
1) Make closes jobserver descriptors and invokes command
2) Command opens something using the same descriptor
3) Command itself runs make, leaving those descriptors open
4) make tries to use those descriptors as jobserver descriptors

To fix this, in a future version of make will will do at least one, and
maybe both, of two things:
A) when we close the file descriptors, update MAKEFLAGS to remove them
so that sub-makes won't try to use them.
B) switch away from simple pipes to using named pipes for jobserver
implementation; then we don't need to worry about leaving descriptors
open

To fix the issue for now in your environment, you'll need to perform
the equivalent of solution (A) in your own environment.

That is, whatever recipe you have that invokes cmdA will need to clean
up MAKEFLAGS; either something simple like:

  run-cmdA:
          MAKEFLAGS= cmdA

Or, if you prefer, you can try to just strip out the flags you don't
want, something like this:

  run-cmdA:
          MAKEFLAGS='$(filter-out --jobserver-auth%,$(MAKEFLAGS))' cmdA

Obviously the details are up to you.