help-make
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Make & Ninja benchmark with 1k, 10k and 100k source files


From: David Rothlisberger
Subject: Re: Make & Ninja benchmark with 1k, 10k and 100k source files
Date: Thu, 10 Nov 2016 00:19:57 +0000

On 9 November 2016 at 16:33, Paul Smith <address@hidden> wrote:
> > Make does try to build depth-first, but it
> > seems to push the targets to the end of a queue if it needed to build
> > their prerequisites first
>
> The details will require a bit of investigation.

I've done some further experiments. Make behaves correctly for serial
builds; I can only reproduce the unexpected behaviour with "-jN".

    $ make
    cc 1/a.o
    cc 1/b.o
    link 1/prog
    cc 2/a.o
    cc 2/b.o
    link 2/prog
    cc 3/a.o
    cc 3/b.o
    link 3/prog

    $ make -j2
    cc 1/a.o
    cc 2/a.o
    cc 1/b.o
    cc 2/b.o
    cc 3/a.o
    cc 3/b.o
    link 1/prog
    link 2/prog
    link 3/prog

I've tested make 3.81, 3.82, 4.1, and git master (4.2.1-5-g7ed37f0).
They all behave the same way.

This has been brought up before[1] but the reporter never investigated
further.

[1]: https://lists.gnu.org/archive/html/help-make/2009-11/msg00099.html

>From looking at debug output and at remake.c I think I can see what's
causing this, and it isn't going to be easy to fix! Probably not worth
it. FWIW I've written an automated test-case (included below).

Cheers,
Dave.

---
>From cd55ca9f6b399e58aeff688ad1bfb1d440808c34 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?David=20R=C3=B6thlisberger?= <address@hidden>
Date: Wed, 9 Nov 2016 20:56:12 +0000
Subject: [PATCH] * tests/scripts/features/depth_first: Add failing test for
 parallel build

Test 0 passes, but Test 1 (with "-j2") fails -- make seems to push
the goal target to the end of a queue if it needed to build its
prerequisites first.

See discussion at:
https://lists.gnu.org/archive/html/help-make/2016-11/msg00004.html
---
 tests/scripts/features/depth_first | 39
++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
 create mode 100644 tests/scripts/features/depth_first

diff --git a/tests/scripts/features/depth_first
b/tests/scripts/features/depth_first
new file mode 100644
index 0000000..22b235e
--- /dev/null
+++ b/tests/scripts/features/depth_first
@@ -0,0 +1,39 @@
+#
-*-perl-*-
+$description = "Test the order in which targets are built.";
+
+$details = "\
+We test that make builds targets in a depth first order
+(dependencies permitting), in both serial and parallel
+builds.
+";
+
+$mk_string = '
+all: A1 A2 A3
+A1: B1 ; @echo $@
+A2: B2 ; @echo $@
+A3: B3 ; @echo $@
+B1:    ; @echo $@
+B2:    ; @echo $@; sleep 0.1
+B3:    ; @echo $@
+';
+
+# TEST 0: Default goal's dependencies are processed left to right, depth
first.
+
+run_make_test($mk_string, '', "B1\nA1\nB2\nA2\nB3\nA3");
+
+# TEST 1: As above, in parallel (2 jobs).
+#
+# Job1 looks at A1, needs to build its dependency B1.
+# Job2 looks at A2, needs to build its dependency B2.
+#
+# Job1 will complete first (because of B2's sleep). It SHOULD now build A1
+# because B1 has been built, but instead it builds B3.
+#
+# N.B. I need --output-sync so that the test isn't racy.
+
+if ($parallel_jobs and exists $FEATURES{'output-sync'}) {
+  run_make_test($mk_string, '-j2 --output-sync', "B1\nA1\nB3\nA3\nB2\nA2");
+}
+
+# This tells the test driver that the perl test script executed properly.
+1;
-- 
2.7.4


reply via email to

[Prev in Thread] Current Thread [Next in Thread]