Re: dd does not handle input from pipes correctly (coreutils-5.93)

[Paul Mielke]

Hi, Paul.

Thanks for your response!  I figured that my bug report would go into a
black hole.  It's a pleasant surprise to get any response at all, let alone
one that is both timely and to the point.

It looks like the behavior of dd on Solaris 10 is even more bizarre.  Or
perhaps the problem is with /dev/urandom.  I would have expected the
file "pad" to be 16384 bytes in size.

I agree that the OpenGroup spec does make it sound as if the behavior
is intended.  The net result is that dd is not really useful when taking
input from a pipe.  Oh, well.

After I sent the bug report, I realized why it was that we suddenly
started seeing this problem in a script that had been working for
quite a while:  we just got a new machine that's a dual cpu box
and are running an MP linux kernel.  The first write to the pipe
wakes up the reader and he actually gets to run, instead of hanging
out to dry on the run queue until the writer fills up the pipe buffer
and blocks.  On a single cpu machine, you would only see the
problem very rarely (if an interrupt happens to cause a reschedule
at just the wrong time).

I guess you can officially close or reject the bug in your database.

Best regards,
Paul

At 10:51 PM 11/17/2005 -0800, Paul Eggert wrote:
>Paul Mielke <address@hidden> writes:
>
> > The problem is this:  if input is from stdin and stdin is a pipe, the
> > routine dd_copy assumes that a short read returned from "iread"
> > means that it has read a partial record.
>
>I think that's what dd is supposed to do.  See, for example,
><http://www.opengroup.org/onlinepubs/000095399/utilities/dd.html>.
>
> > The easiest way to see the bug is to do something like:
> >
> > dd if=/dev/urandom of=foo count=3
> > dd if=/dev/urandom of=pad bs=16k count=1
> >
> > cat foo pad | dd of=foo.padded bs=16k count=1
> >
> > When the failure occurs, foo.padded ends up being shorter than 16k.
>
>Solaris 10 dd does something similar to GNU dd (so if GNU dd is wrong
>at least it's in good company :-):
>
>575-otter $ dd if=/dev/urandom of=foo count=3
>3+0 records in
>3+0 records out
>576-otter $ dd if=/dev/urandom of=pad bs=16k count=1
>0+1 records in
>0+1 records out
>577-otter $ cat foo pad | dd of=foo.padded bs=16k count=1
>0+1 records in
>0+1 records out
>578-otter $ ls -l foo pad foo.padded 
>-rw-rw-r--   1 eggert   faculty     1536 Nov 17 22:47 foo
>-rw-rw-r--   1 eggert   faculty     1536 Nov 17 22:47 foo.padded
>-rw-rw-r--   1 eggert   faculty     1040 Nov 17 22:47 pad
>

Paul Mielke                         address@hidden
BroadOn Communications    (650) 739-5377