Re: running parallel make jobs and avoiding jumbled/garbled output?

[Paul Smith]

On Fri, 2008-02-22 at 18:39 +0100, Per Jessen wrote:
> > However, it sounds to me like the OP is seeing this issue even with
> > output to stdout, which is already line-buffered (or is supposed to
> > be). So, I think the issue might be more complex than that.
> 
> No, if I run the same 'make -j4' job in a terminal, there's no problem. 
> No garbling and no truncation of output. 

Right, because when you run it in a terminal the output of your compiler
is set to line buffered, because that's how libc is defined (stdout is
line buffered when writing to terminals, unbuffered when writing
somewhere else).

> > I'm pretty sure that line buffering settings are user-space features,
> > not process features, and as such they are not inherited by
> > sub-processes but rather determined anew for every process that
> > starts, based on its terminal settings, etc.
> 
> Hmm, when a new process is forked, I'm pretty certain the default is for 
> it to continue using whatever stdin/out/err settings the parent process
> was using.

Nope, I really don't think that's correct.  Remember the buffering is a
100% user-space feature: the entirety of the stdio library (FILE*,
setbuf, etc.) is ALL a construction of the C runtime library.  It's all
implemented on top of the system calls like read(2) and write(2), which
are the only things the kernel knows about.

Because buffering is all user-space and an attribute of a FILE*, NOT an
attribute of a file descriptor, it's not something that the kernel can
propagate to a new process when you fork(2).  There are many attributes
of fd's that ARE propagated such as close-on-exec etc., but I'm quite
confident that stdio buffering is not one of those.

So, there's really nothing make can do about this.  You either have to
change the code in the compilers, etc. to explicitly set line buffering,
OR you'll have to invoke make in such a way that stdout is attached to a
terminal, or else fake it into thinking that it is by attaching it to a
pty, using expect or screen or some similar program.

-- 
-----------------------------------------------------------------------------
 Paul D. Smith <address@hidden>                 http://make.mad-scientist.us
 "Please remain calm--I may be mad, but I am a professional."--Mad Scientist