coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RFC: enable potential checksum output buffering improvements


From: Pádraig Brady
Subject: RFC: enable potential checksum output buffering improvements
Date: Sun, 5 Aug 2012 19:31:09 +0100

Perhaps useful for some future release?

I noticed one of my scripts for processing checksums
of many small files was using lots of CPU.
In particular 'sed' was showing up in top.
I replaced 'sed' with 'tee dbg | sed' and then
tee started using the CPU.
This indicated that the previous process in the pipe
(md5sum) was causing excessive context switching.
Note the Linux kernel behavior of encouraging wakup
of the process reading from a pipe when the pipe is written to,
does seem sensible to me.
I confirmed reduced CPU usage by removing the
setvbuf(..,_IOLBF,..) call in md5sum.

Now we only use this to ensure atomic line output,
and the existing setvbuf() interface seems rich enough
to express just this requirement, by also specifying a
buffer size to use with _IOLBF.  So I proposed such
an augmentation of the stdio implementation in glibc here:
http://sourceware.org/bugzilla/show_bug.cgi?id=14433

This patch would enable such logic, but could
be applied at any stage.

cheers,
Pádraig.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]