[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Using shell wrapper for descrambling parallel make output

From: Atte Peltomäki
Subject: Re: Using shell wrapper for descrambling parallel make output
Date: Mon, 14 Nov 2011 11:29:04 +0200
User-agent: Mutt/1.5.20 (2009-06-14)

On Fri, Nov 11, 2011 at 10:24:25AM -0500, David Boyce wrote:
> On Fri, Nov 11, 2011 at 9:26 AM, Atte Peltomäki <address@hidden> wrote:
> > Hi,
> >
> > As you know, running parallel builds with '-jX' makes the shell output
> > difficult to read, since output from parallel jobs are mixed. To remedy
> > this, the use of a buffering shell wrapper has been suggested:
> >
> >
> >
> > I liked the idea, but implementation was lacking so I rewrote the
> > wrapper.
> That's a good thing to do but the work may be duplicative. See
> Take a look and tell me if you think
> it's worth comparing notes in order to get the best of both.
> > If I have time and interest, I will look into implementing the wrapper
> > functionality into make itself. Don't hold your breath, though.
> Similarly, almost the same thing has already been done and a patch
> exists against 3.82 which I hope will make it into 3.83. See
> There's also a long thread in the
> bug-make archives from around the same time (4/2011) discussing it.

Nice work. Your implementation seems much more refined than mine. Only
one thing catches my attention; your version doesn't seem to properly
preserve the original line ordering between stdout and stderr. I suggest
solving this as I did:

- Don't write the output to files, just use a pipe, change vfork() to 
  fork() and read child output from the parent using the pipe.

- Use a simple struct as a buffer for the data: one member of the struct
  is an int that is 0 for stdout and 1 for stderr, other member is a one
  line of output (char *). 

- Use a temporary buffer to read() from pipe, then dynamically allocate
  only needed amount of memory per line to the the struct char* member.

See my code source for details.

Atte Peltomäki
     address@hidden <>
"Your effort to remain what you are is what limits you"

reply via email to

[Prev in Thread] Current Thread [Next in Thread]