Parallel grep example

[Harry B]

Hello,

I tried the parallel grep example on a sample file  (1.4G and 3,153,199 lines). The parallel grep (http://www.gnu.org/software/parallel/man.html#example__parallel_grep)  is considerably slower - I am trying to see where the bottle neck is.  Is there an easy way for me to guess which way would be better based on the size of the file?

# Regular grep
$ time cat testfile | grep -F test_pattern | wc -l
117

real    0m1.208s
user    0m0.543s
sys     0m1.704s

$ time cat testfile | parallel --pipe grep -F test_pattern | wc -l
117

real    0m18.815s
user    0m11.807s
sys     0m16.945s

The test was repeated multiple times to rule out disk speed. It was however similarly slower the first time I ran it as well.

NOTE: It is not that i am trying to make a 1 second grep any faster :). I am just trying to find why the 'parallel --pipe' was 10x slower. I would have expected a similar or slightly worse performance for a 1.4G file.

-- 
Harry