[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: parallel make help

From: Bob Friesenhahn
Subject: Re: parallel make help
Date: Thu, 10 Apr 2008 11:50:15 -0500 (CDT)

On Thu, 10 Apr 2008, Bob Rossi wrote:

If I moved everything to a single at the top level, would
that improve the speed of things so that it could continue to build
in parallel?

Yes. If you convert your project to a non-recursive build, then it is very likely that considerably more parallelism will be exposed.

In any large build, there are certain points where all parallism must cease and it is necessary to wait for one task to complete. Typically this occurs whenever there is a link of a library that other targets need. Libtool convenience libraries are one of these points. It also occurs whenever a directory is entered via the simplistic SUBDIR mechanism. With the simplistic SUBDIR mechanism, only targets built in the subdirectory being built are candidates for parallel compile. With a non-recursive build you can eliminate use of the SUBDIR mechanism and convenience libraries.

Depending on how you structure your libraries and apportion source modules between common libraries and final executable code, you can achieve massive parallism with a non-recursive build.

Users with multiple cores and lots of memory will be happy since there is almost linear speedup. It remains to be seen if distributing compile jobs to other machines speeds things up since there is a cost associated with distributing compile jobs. If your source modules are quick to compile, there may not be any win.

If it is in fact faster, like I'm imagining it might be, is there any
way to make the " in each subdir" approach be just as fast?

No. However, you can still gain the maintenance benefits of " in each subdir" by using the Automake include mechanism with non-recursive build.

Bob Friesenhahn
GraphicsMagick Maintainer,

reply via email to

[Prev in Thread] Current Thread [Next in Thread]