help-make
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Looking for help improving performance (advice or potential contract pos


From: Phil Grabsky
Subject: Looking for help improving performance (advice or potential contract position)
Date: Wed, 1 Oct 2014 13:44:02 -0400

Greetings all,
We've been making an effort to optimize our builds and we seem to be
hitting the limit of what we can accomplish with makefile cleanup.  I've
turned around some significant improvements by using techniques like
simplifying makefiles where possible, converting recursively-expanded
variables to simple variables, eliminating shell commands, etc, but we're
still seeing slow performance from Make itself.

Our build is extremely parallelizable and we run with -j (unlimited) to run
as many things as possible in parallel.  We use a home-grown job
distribution tool to send individual tasks to clients on other machines for
execution, so there's very little load on the machine running Make itself.
What we see is that if we have more than 30 or so clients connected to the
build Make is often unable to generate tasks fast enough to keep them busy.

We brought in Electric Cloud to do an evaluation of their make-compatible
accelerator tool and found that it was able to instantly saturate as many
workers as we could throw at it and keep them busy for the duration of the
build, cutting the overall time nearly in half.  Unfortunately, their
approach introduces a number of changes that would require significant
reworking of our build system, and more importantly they want quite a lot
of money for it (enough to hire additional senior developers for our team).

Our management doesn't want us taking time to learn the internals of make
and see if we can improve it ourselves, but they are willing to throw money
at the problem, either by offering some sort of donation/bounty for
performance improvements or directly hiring an expert on a contract basis;
any improvements achieved would of course be submitted back to the
community as patches.

Is there an established place to post this sort of offer? fossfactory.org
seems like what I'm looking for, but it doesn't look like it's terribly
active.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]