[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How do parallel builds scale?

From: Howard Chu
Subject: Re: How do parallel builds scale?
Date: Thu, 03 Mar 2011 08:25:58 -0800
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:2.0b11pre) Gecko/20110130 Firefox/4.0b11pre SeaMonkey/2.1b2pre

Ludovic Courtès wrote:

There was recently discussion as to how configure could be parallelized.

I ran a series of build time measurements on a 32-core machine, with
make -jX, with X in [1..32], and the results are available at:

It's been my experience that compile-oriented builds tend to be I/O bound and generally have excess CPU capacity in the range of 30-40%. As such I always use make -jX where X is number of cores * 1.5, which tends to keep the idle CPU percentage below 5%.

There are also insightful per-package details:

There are packages whose configuration phase is noticeably longer than
the build time.

Yes. I've seen several projects where the autotool files are larger than the actual program source code. IMO, any time your build overhead takes up more than 1% of your total project size, you've done something wrong.

OTOH, packages with a long build time (with a sequential time in
boldface) don’t scale well either, but that is not configure’s fault.

If only more people would learn to write proper Makefile dependencies...

  -- Howard Chu
  CTO, Symas Corp. 
  Director, Highland Sun
  Chief Architect, OpenLDAP

reply via email to

[Prev in Thread] Current Thread [Next in Thread]