[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Parallel building of Linux Kernel is broken

From: Mike Shal
Subject: Re: Parallel building of Linux Kernel is broken
Date: Mon, 10 Sep 2018 12:24:21 -0400

On Mon, Sep 10, 2018 at 4:18 AM Masahiro Yamada <address@hidden> wrote:

Seems no more feedback for this regression report.

OK, the Linux kernel build system is too complicated.
So, I have come back with a much simpler test-case.

Here, the test-case is only 2 makefiles, less than 50 lines.

Please take a look this problem.

As I already reported, the git-bisect points to

commit 2b8e3bb23f96c2458818f011593557d3353dade3
Author: Paul Smith <address@hidden>
Date:   Mon Jan 2 14:08:54 2017 -0500

    Clean up close-on-exec, particularly with jobserver pipes.

I attached the test case below.

For convenience, this test-case is available from my GitHub repository as well:

[Test Case]


# If MAKECMDGOALS contains two or more targets, handle them one by one.
ifneq ($(word 2,$(MAKECMDGOALS)),)
PHONY += $(MAKECMDGOALS) __build_one_by_one

$(filter-out __build_one_by_one, $(MAKECMDGOALS)): __build_one_by_one

        set -e; \
        for i in $(MAKECMDGOALS); do \
                $(MAKE) -f Makefile $$i; \


ifeq ($(MAKECMDGOALS),config)

config: FORCE
        touch .config


include auto.conf

PHONY += all
        echo all

auto.conf: .config
        $(MAKE) -f Makefile.config syncconfig



----------------------(Makefile END)---------------------------

        touch auto.conf
----------------------(Makefile.config END)---------------------------

It looks like the patch in question changed the default state of the jobserver tokens to be not inherited. Since this Makefile generates an included file (auto.conf), the 'make all' invocation creates that file and then re-invokes itself. However, the re-invoking of make happens in main.c instead of posixos.c, and is not wrapped in jobserver_pre_child/post_child which are responsible for updating the jobserver fd inheritance. I'd guess main.c also needs to call fd_inherit on the jobserver tokens (or use jobserver_pre_child?) before the re-invocation call. The following hack seems to fix the issue:

diff --git a/main.c b/main.c
index 5dd539b..83f30f8 100644
--- a/main.c
+++ b/main.c
@@ -2446,7 +2446,9 @@ main (int argc, char **argv, char **envp)
           if (stack_limit.rlim_cur)
             setrlimit (RLIMIT_STACK, &stack_limit);
+      jobserver_pre_child(1);
           exec_command ((char **)nargv, environ);
+      jobserver_post_child(1);
           free (aargv);


reply via email to

[Prev in Thread] Current Thread [Next in Thread]