[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Updated patch (was: Re: improving the -l option)

From: Sven C. Dack
Subject: Updated patch (was: Re: improving the -l option)
Date: Tue, 22 Aug 2017 17:21:42 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1


here is an updated version of the patch improving the performance of the -l option. It's a diff against the current git master.

The update includes O_CLOEXEC for opening the file descriptor so it won't "bleed" an open file descriptor to child processes. The configure script checks for a useable /proc/loadavg file, but the result can be overwritten manually with --enable-proc-loadavg={yes,no}.



On 09/10/16 19:58, Sven C. Dack wrote:

I have created a patch to improve make's behaviour with regards to parallel execution and system load.

On Linux do the load averages refresh only every 5 seconds. Further do load averages only ever represent past but not present load. As a consequence does this lead to idle times during parallel builds where make is waiting for the load averages to come down. It can also spawn excessive amounts of processes (although it tries to balance it with its own heuristic).

A better way to achieve a stable load is to use the active number of running processes & threads over of the load averages. This number can be found in /proc/loadavg on Linux and is updated in real-time. An example:

$ cat /proc/loadavg
0.00 2.72 6.89 1/404 1890

Here it shows 1 (out of 404) running processes. The patch allows make to use the 4th value from /proc/loadavg.

The overall improvement for a parallel build on an 8-core CPU:

-j -l 24 (old): 269.993128122 seconds time elapsed
-j -l 24 (new): 231.300406406 seconds time elapsed

In comparison:

-j 24: 230.482726977 seconds time elapsed

The behaviour of the -l <value> option with the patch is now much closer to that of the -j <value> option and shows almost identical build times for an empty system.

The test suite passes successfully with the patch.

You'll find two patch files against make-4.2.1 in the attachment. One for configure.ac (includes a test for /proc/loadavg usability) and the other for job.c (modifies the function load_too_high()).


Bug-make mailing list

Attachment: proc-loadavg.patch
Description: Text Data

reply via email to

[Prev in Thread] Current Thread [Next in Thread]