Re: Proposed new feature for bash: unbuffered pipes

[Dale R. Worley]

The crux of the problem, IMHO, is to look at it from the right angle:

Occasionally, the user desires that I/O through certain pipes should be
unbuffered, that is, the stdio stream(s) that write into the pipe should
be unbuffered, rather than the default block-buffered.  These are
situations where the user can tell that it is desirable to sacrifice
efficiency to gain low latency, whereas the writer of the program that
the user is running could not predict the needs of this particular
invocation.  (This is where Ulrich Drepper was incorrect; the writer of
the program doesn't know when this is desirable.)

This facility is best activated by a feature in bash, as the need arises
from the relationship between two processes.

The ideal way to implement this would be for the kernel to provide two
flavors of pipe, one designated buffered and one designated unbuffered,
and according to the user's instructions, bash would allocate an
unbuffered pipe when it was needed.  The kernel itself would treat the
two types of pipe in the same way, but stdio would create buffered
streams for fd's that were opened on buffered pipes and it would create
unbuffered streams for fd's that were opened on unbuffered pipes.

However, getting a coordinated change into the kernel seems like it
would be quite difficult.

So an alternative implementation is to have a way for bash to notify
processes it creates that certain pipes (identified by their dev/ino
numbers) should have streams that are unbuffered.  Pretty much the only
way to do this is to have a conventional environment variable for
communicating this information.

Of course, either of these solutions requires changes to glibc, but
then, if you're going to modify buffering behavior of streams, you are
going to have to modify glibc.

Dale