GNU Parallel Bug Reports inefficiency of --pipe -L

[Martin Frith]

Hello:

The NEWS file mentions that '-L n --pipe' is useful for fastq.  I tried this, and it works, but extremely slowly.  On the other hand, using '-N' instead of '-L' works and is much faster.

I'm using GNU parallel 20120822.  Here is how to reproduce the problem:

seq 100000 > fake.fastq

# This works and is fast:

time parallel --pipe -N10000 cat < fake.fastq > out1

# These work but are extremely slow:

time parallel --pipe -L10000 cat < fake.fastq > out2a

time parallel --pipe -L4 cat < fake.fastq > out2b

time parallel --pipe -L4 --block 1M cat < fake.fastq > out2c

(As you can see, I'm also not certain how -L is supposed to be used.)

I hope this is useful,

Martin Frith

http://www.cbrc.jp/~martin/