[bug #51462] Double-colon dependencies are built serially with parallel make

[anonymous]

URL:
  <http://savannah.gnu.org/bugs/?51462>

                 Summary: Double-colon dependencies are built serially with
parallel make
                 Project: make
            Submitted by: None
            Submitted on: Thu 13 Jul 2017 09:29:18 PM UTC
                Severity: 3 - Normal
              Item Group: Bug
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
       Component Version: SCM
        Operating System: None
           Fixed Release: None
           Triage Status: None

    _______________________________________________________

Details:

I noticed that some builds are significantly slower after switching from make
4.0 to 4.2.1.  I tracked it down to what seems to be a parallel make problem
with double-colon prerequisites.  In particular, when there are multiple
double-colon prereqs for a target, each of those prereqs seems to be
considered serially, although their dependencies are built in parallel.  It
may be easier to explain with an example.

Test makefile:


default: all

# define a rule to build 'file$(1)', as an intermediate for 'target$(2)'.
#
# The recipe just prints the time, target name, and information about what
# target it's for, sleeps for $(1) seconds, then touches the file.
#
# It also adds a dependency for 'target$(2)'.
define FILE_RULE

file$(1):
        @echo At `date +%k:%M:%S` building $$@, for $(2)
        @sleep $(1)
        @touch $$@

target$(2): file$(1)

endef

# define a rule to build 'target$(1)'.
#
# The recipe just prints the time and touches the file.
#
# It also adds a double-colon dependency for the 'all' phony target, so that
# 'target$(1)' will get built when 'all' is the goal.
define TARGET_RULE

target$(1):
        @echo At `date +%k:%M:%S` building $$@
        @touch $$@

all:: target$(1)

endef

# Declare a few arbitrary dependency trees using the convenience macros
above.
# The first two have three dependent files each, the last one has six.
$(eval $(call FILE_RULE,1,1))
$(eval $(call FILE_RULE,2,1))
$(eval $(call FILE_RULE,3,1))
$(eval $(call TARGET_RULE,1))

$(eval $(call FILE_RULE,4,2))
$(eval $(call FILE_RULE,5,2))
$(eval $(call FILE_RULE,6,2))
$(eval $(call TARGET_RULE,2))

$(eval $(call FILE_RULE,7,3))
$(eval $(call FILE_RULE,8,3))
$(eval $(call FILE_RULE,9,3))
$(eval $(call FILE_RULE,10,3))
$(eval $(call FILE_RULE,11,3))
$(eval $(call FILE_RULE,12,3))
$(eval $(call TARGET_RULE,3))

clean:
        rm -rf file* target*

.PHONY: all clean


With GNU make 4.0 the output of 'time make -j' after 'make clean' is:

At 14:08:02 building file4, for 2
At 14:08:02 building file6, for 2
At 14:08:02 building file3, for 1
At 14:08:02 building file2, for 1
At 14:08:02 building file1, for 1
At 14:08:02 building file5, for 2
At 14:08:02 building file9, for 3
At 14:08:02 building file11, for 3
At 14:08:02 building file7, for 3
At 14:08:02 building file12, for 3
At 14:08:02 building file8, for 3
At 14:08:02 building file10, for 3
At 14:08:05 building target1
At 14:08:08 building target2
At 14:08:14 building target3

real    0m12.026s
user    0m0.008s
sys     0m0.004s


Note that all 'fileN' recipes begin at the same time, and the dependent
'targetN' recipes begin as soon as their respective prereqs are complete.

The same with GNU make 4.2.1:

At 14:09:25 building file1, for 1
At 14:09:25 building file2, for 1
At 14:09:25 building file3, for 1
At 14:09:28 building target1
At 14:09:28 building file4, for 2
At 14:09:28 building file5, for 2
At 14:09:28 building file6, for 2
At 14:09:34 building target2
At 14:09:34 building file7, for 3
At 14:09:34 building file8, for 3
At 14:09:34 building file9, for 3
At 14:09:34 building file10, for 3
At 14:09:34 building file11, for 3
At 14:09:34 building file12, for 3
At 14:09:46 building target3

real    0m21.052s
user    0m0.004s
sys     0m0.004s


Note how the prereqs for target1 are built, then once that's complete the
prereqs for target2, and then finally target3.  Each set doesn't start until
the previous has completed.


I bisected the GNU make repository and found that this is caused by the
commit:
[SV 44742] Fix double-colon rules plus parallel builds.
(9bb994e8319c2b153cd3d6d61e2c2882895e7c3a in the git repo).

The 4.2.1 behavior also occurs at the latest commit from source control
(e2ebea35f11059e in the git repo).




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?51462>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/