[Fwd: grep malfunctions under heavy load and non-matching lines do not get filtered out]

[Sean Lee]

They say I should submit this one to you.

Regards
Sean

-----Forwarded Message-----

> From: Sean Lee <address@hidden>
> To: address@hidden
> Cc: address@hidden
> Subject: grep malfunctions under heavy load and non-matching lines do not get 
> filtered out
> Date: 11 Mar 2003 20:07:59 +0800
> 
> Configuration Information [Automatically generated, do not change]:
> Machine: i686
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='i686'
> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu'
> -DCONF_VENDOR='pc' -DSHELL -DHAVE_CONFIG_H  -I.  -I. -I./include -I./lib
> -D_GNU_SOURCE  -O2 -march=i386 -mcpu=i686
> uname output: Linux kate.erexi.com 2.4.18-17.8.0 #1 Tue Oct 8 11:25:39
> EDT 2002 i686 i686 i386 GNU/Linux
> Machine Type: i686-pc-linux-gnu
> 
> Bash Version: 2.05b
> Patch Level: 0
> Release Status: release
> 
> Description:
>       Sometimes non-matching lines don't get filtered out by grep.
> Sample output of my utility:
> {MD4}Uunp5PHNbRq8eKqGbSpVCQ==
> {MD5}Gh3JHJBzJcaScd3wyUS8cg==
> {RMD160}43qIaB6q/fR6Woa0w5mGk+ezE6s=
> {SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=
> I call grep like this: `./digest $pass | grep SHA` from Perl or bash.
> I get about 10 malfunctions on 30K of entries.
> 
> Malfunction example:
> password: {RMD160}43qIaB6q/fR6Woa0w5mGk+ezE6s=
> {SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=
> 
>       Digest is a qmail utility for generating passwords (MD4, MD4, SHA..)
> with each run producing four lines of output.
>       It could be the digest utility,maybe sometimes it does not insert
> newline.
>       At this moment I'm unsure if this is a digest or a bash bug or neither.
> 
> Repeat-By:
> Generate a random CSV file (30K entries, for example), with two columns,
> such as "user,password", then feed it to a script (Perl or bash) to
> generate passwords and obtain only the SHA password. 
> Sample result of digest: ./digest pass
> {MD4}Uunp5PHNbRq8eKqGbSpVCQ==
> {MD5}Gh3JHJBzJcaScd3wyUS8cg==
> {RMD160}43qIaB6q/fR6Woa0w5mGk+ezE6s=
> {SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=
> Call it like this: `./digest $pass | grep SHA`
> Then grep the resulting file for MD4 or MD5 to find problematic lines.
> In my case:
> 
> Attached is digest I use and the source. For more info, www.qmail.org.
> 
> Fix:
>       No fixes, I write another script to fix resulting file.
>

digest
Description: application/executable-binary

digest.c
Description: Text Data