[debbugs-tracker] bug#23837: closed (grep 2.7 EXIT STATUS inconsistency)

GNU grep team,

I’ve found some inconsistency with the EXIT STATUS processing of grep 2.7. All data used in the test is shown below, along with the command line arguments to grep. Everything needed to repeat my test cases is described in this email. Below are the details.

Version Information:

> grep --version

grep (GNU grep) 2.7

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

Details:

From the Manual page of grep, the text of the EXIT STATUS follows:

EXIT STATUS

Normally, the exit status is 0 if selected lines are found and 1 otherwise. But the exit status is 2 if an error occurred, unless

the -q or --quiet or --silent option is used and a selected line is found. Note, however, that POSIX only mandates, for programs

such as grep, cmp, and diff, that the exit status in case of error be greater than 1; it is therefore advisable, for the sake of

portability, to use logic that tests for this general condition instead of strict equality with 2.

Below are two test cases that show the EXIT STATUS from grep is not necessarily described as “Normally, the exit status is 0 if selected lines are found and 1 otherwise”.

· Note1: $GREP_EXCLUDE used below is simply a shell variable pointing to a file containing the lines to exclude from the output, and set with a command similar too: GREP_EXCLUDE=./testing.grep_exclude

· Note2: During the testing below, the *only change* was to the contents of the $GREP_EXCLUDE file.

· Note3: The input data remained the same between test cases.

· Note4: The input data is ascii text, with each line terminated with a newline. Each line in the input file begins with the pattern: “n,n,” and continues to the EOL, or newline character. There are 7 lines in the input file.

Test case one will show the return code from grep when the -ivEF “exclude” removes *all* data records from the input. Notice, the input file is fed to grep, and there’s no output -- in other words, all records were (or “selected to be”) excluded. The return code (or EXIT STATUS) is 1.

Thu Jun 23 22:12:46 UTC 2016

Test Case 1: grep exclude file with filters that will exclude all data from input

hpehrshdev:~/tmp> cat mjodata.txt | grep -ivEf $GREP_EXCLUDE

hpehrshdev:~/tmp> echo $?

hpehrshdev:~/tmp> cat mjodata.txt

0,5,"2016-06-17 16:30:54.404000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 16:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=1, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 16:30:53.0000000."

0,5,"2016-06-17 15:30:54.158000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 15:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 15:30:53.0000000."

0,5,"2016-06-17 14:30:53.780000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 14:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 14:30:53.0000000."

0,5,"2016-06-17 13:30:54.906000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 13:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 13:30:53.0000000."

0,5,"2016-06-17 12:30:54.665000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 12:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 12:30:53.0000000."

77,4,"2016-06-17 12:30:56.582000000","The disk usage of this database has exceeded 7498 GB. See system view M_DISK_USAGE for more details."

69,4,"2016-06-17 12:33:20.228000000","Automatic log backup is disabled."

hpehrshdev:~/tmp>

hpehrshdev:~/tmp> cat $GREP_EXCLUDE

^69,4,.*Automatic log backup is disabled.*$

^77,4,.*The disk usage of this database has exceeded.*See system view M_DISK_USAGE for more details.*$

^0,5,.*301 unique constraint violated: TrexUpdate failed on table

hpehrshdev:~/tmp>

Test case two will show the return code from grep when the -ivEF “exclude” removes *only some* of the data records from the input. Notice, the input file is fed to grep, and output is returned -- in other words, some records were (selected to be) excluded, but some of the records did not get excluded (because there was no filter for them). The return code (or EXIT STATUS) is 0.

Thu Jun 23 22:15:35 UTC 2016

Test Case 2: grep exclude file with filters that will exclude some of the data from input

hpehrshdev:~/tmp> cat mjodata.txt | grep -ivEf $GREP_EXCLUDE

hpehrshdev:~/tmp> echo $?

hpehrshdev:~/tmp> cat mjodata.txt

77,4,"2016-06-17 12:30:56.582000000","The disk usage of this database has exceeded 7498 GB. See system view M_DISK_USAGE for more details."

69,4,"2016-06-17 12:33:20.228000000","Automatic log backup is disabled."

hpehrshdev:~/tmp> cat $GREP_EXCLUDE

^69,4,.*Automatic log backup is disabled.*$

^77,4,.*The disk usage of this database has exceeded.*See system view M_DISK_USAGE for more details.*$

hpehrshdev:~/tmp>

Issue:

In scripting, it is normal to process the EXIT STATUS of grep in such a way that an EXIT STATUS of 0 (zero) indicates that matches to the grep pattern were found. In *both of* the above test cases matches *were* found, yet the EXIT STATUS was not consistently 0. In test case 1, *all* records were matched, yet the EXIT STATUS was 1. With this behaviour of grep, it is * not possible to correctly* perform the following simple logic (bash syntax):

typeset -i RC=0

cat mjodata.txt | grep -ivEf $GREP_EXCLUDE

RC=$?

if [ $RC -eq 0 ]; then

#Grep found matches

#Do something…

else

#No match found, or possibly some error -- needs more checking

Michael J. O'Brien
Consultant Architect, AMS ABS Global Practice

address@hidden

+1 650 258 2628 Office
+1 916 335 3010 Mobile

US/Pacific time zone
hpe.com

From:	GNU bug Tracking System
Subject:	[debbugs-tracker] bug#23837: closed (grep 2.7 EXIT STATUS inconsistency)
Date:	Fri, 24 Jun 2016 08:38:01 +0000