Re: line buffering in pipes (was: RFC: Safely using xargs -P$NUM children's output? Need a new tool?)

[Egmont Koblinger]

Hi Assaf,

On Thu, May 2, 2019 at 9:14 PM Assaf Gordon <address@hidden> wrote:

> The easiest way to avoid that is to use "stdbuf" (from coreutils),
> forcing a flush after each line. Assuming the lines are short enough
> (and file's output should be short enough), it should work:
>
>     find /usr -print0 | xargs -0r -P199 -n16 stdbuf -oL file | ...

I don't think this is robust enough. If many "stdbuf -oL file"
processes decide to produce a reasonably sized output pretty much at
the same time, it might still suddenly clog the pipe and result in a
short write in one of them. Or am I missing something?

cheers,
egmont