Re: weird interaction betwixt cat & dd (?) w/coreutils-6.9

[Andreas Schwab]

Linda Walsh <address@hidden> writes:

> I started with 2 files about 250MB in length (random, compressed data)
> and wanted to create a 256MB file.  I thought to concatenate the
> 256MB files with cat and use dd to read a fixed amount to an output
> file:
>    cat testfile-250M testfile.gz | dd of=250MB bs=1M count=256
>
> Is something "wrong" with the above"?...because it didn't work.

In a nutshell, use "head -c 256M" instead.

dd reads data in units of the input block size.  In the course it tries
to read up to the input block size in one go (one read syscall).  If it
gets less than that it continues with the partial block, but counts it
as if it was a full block.  A pipe only guarantees $(ulimit -p) * 512
bytes (4Kb typically) of data atomically, anything bigger can be
delivered in several chunks.  Thus for a block size of 256MB the
likelyhood of a partial block is rather high.

Andreas.

-- 
Andreas Schwab, SuSE Labs, address@hidden
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."