[debbugs-tracker] bug#23892: closed (grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.)

[GNU bug Tracking System]

Your message dated Mon, 4 Jul 2016 13:05:25 -0700
with message-id <address@hidden>
and subject line Re: bug#23892: grep is not "grepping" from grep-2.23-1 
(archlinux) with external fixed patterns file.
has caused the debbugs.gnu.org bug report #23892,
regarding grep is not "grepping" from grep-2.23-1 (archlinux) with external 
fixed patterns file.
to be marked as done.

(If you believe you have received this mail in error, please contact
address@hidden)


-- 
23892: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=23892
GNU Bug Tracking System
Contact address@hidden with problems

--- Begin Message --- Subject: grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file. Date: Mon, 4 Jul 2016 15:57:18 +0200

hi,

I've a big (3.3Go) gzipped file which comes from nsrl with fields separated by one tabulation :

$ zcat nsrlfiletxt.gz | head -2
sha-1    md5    crc32    filename    filesize    productcode    opsystemcode    specialcode
000000206738748edd92c4e3d2e823896700f849    392126e756571ebf112cb1c1cdedf926    ebd105a0    i05002t2.pfb    98865    3095    win

I've a file with fixed patterns (windows only from field 7 opsystemcode) :

$ cat win.os
2000 sp 4
2ksp3
dos
...
xp sp2
xphomeedw/sp2
xpprofessw/sp2

my os is :

$ uname -a
Linux arch 4.4.14-1-lts #1 SMP Fri Jun 24 21:35:25 CEST 2016 x86_64 GNU/Linux

and grep is :

$ grep --version
grep (GNU grep) 2.25
...

$ pacman -Q grep
grep 2.25-2

when I try this :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed 's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,4k 0:00:00 [ 776k/s] [ <=> ]

only 59.4k lines are processed, with no error :-( !
(sed is used on win.os to match only on field and pipe view is used to show progess)

I downgrade to grep 2.24 :

# pacman -U /var/cache/pacman/pkg/grep-2.24-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed 's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,4k 0:00:00 [ 863k/s] [ <=> ]

again, only 59.4k lines are processed, with no error :-( !

I downgrade to grep 2.23 :

# pacman -U /var/cache/pacman/pkg/grep-2.23-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed 's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,1k 0:00:00 [ 823k/s] [ <=> ]

only 59.1k lines are processed, with no error :-( !

I downgrade to grep 2.22 :

# pacman -U /var/cache/pacman/pkg/grep-2.22-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed 's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
 157M 0:04:36 [ 567k/s] [ <=> ]

all the 157M of lines are well processed :-) !

so I think there's a bug introduced with grep 2.23...

regards.

--- End Message ---

> --- Begin Message ---
>
>
>
> Subject: 
>
> Re: bug#23892: grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.
>
>
>
>
> Date: 
>
> Mon, 4 Jul 2016 13:05:25 -0700
>
>
>
>
> tags 23892 notabug
> thanks
>
> [I've re-added the bug-tracking address to record that this was not a
> bug and that the issue auto-created by your email is closed. ]
>
> On Mon, Jul 4, 2016 at 11:56 AM, Pascal <address@hidden> wrote:
> > that's right, with LANG=C before grep : all lines are processed :-)
>
> Use LC_ALL=C, not LANG=C. The latter is not portable, while the former is.
>
> > but why it was good with grep 2.22 ?
>
> We discovered bugs -- triggered by e.g., invalid multibyte characters --
> that could cause a segfault or an infinite loop that were present in 2.22,
> and to fix them, we had to make grep more strict.
>
>
> --- End Message ---