Re: [Zutils-bug] zgrep performance long line

[Antonio Diaz Diaz]

Walter Anema wrote:
> I think that your messages_nnl.gz can be processed faster with
> $ time zcat messages_nnl.gz | fold -s -w10000 | grep -o connect  | wc

Folding makes it slightly slower.


> I managed to create another file without private data using the script
>
> i=0
> j=0
> while [ $i -lt 10 ]; do
>     while [ $j -lt 8000000 ]; do
>        printf "%s "  $RANDOM
>        ((j++))
>     done
>     printf "%s" "connect"
>     ((i++))
> done>  largefile

I have tried the code above (which, BTW, puts the 10 "connect" together 
at the end of the file. Not sure that is what you intended) and still I 
can't see any differences between gzip and zutils. Take into account 
that the times in my last mail were from my slow machine (AMD K6-2 450 
MHz). On my desktop machine both gzip and zutils take about 1.2s to 
zgrep largefile.gz.


Best regards,
Antonio.