Re: tests/yesno.sh, kinds of line (matching/non-matching, selected/rejected)

[Julian Foad]

CONTEXT LINES AND MAX-COUNT

Charles Levert wrote:
> Notice that after my private fixes, test #11,
> #27, and #28 of tests/yesno.sh still fail.
>
> ========================================
> Test #11:  { ../src/grep -F -n -b -m 5 -C 1 yes; echo "?$?"; sed 's!^!X!'; }
>   output:  "2-10-[B02 no ]/3:20:[C03 yes]/4:30:[D04 yes]/5:40:[E05 yes]/6-50-[F06 no 
> ]/7-60-[G07 no ]/8:70:[H08 yes]/9:80:[I09 yes]/10-90-[J10 no ]/?0/X[J10 no ]/X[K11 no 
> ]/X[L12 no ]/X[M13 yes]/X[N14 yes]/"
>   expect:  "2-10-[B02 no ]/3:20:[C03 yes]/4:30:[D04 yes]/5:40:[E05 yes]/6-50-[F06 no 
> ]/7-60-[G07 no ]/8:70:[H08 yes]/9:80:[I09 yes]/?0/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13 
> yes]/X[N14 yes]/"
>     FAIL
>  Test #27:  { ../src/grep -F -n -b -m 2 -v -C 1 yes; echo "?$?"; sed 's!^!X!'; }
>   output:  "1:0:[A01 no ]/2:10:[B02 no ]/3-20-[C03 yes]/?0/X[C03 yes]/X[D04 
> yes]/X[E05 yes]/X[F06 no ]/X[G07 no ]/X[H08 yes]/X[I09 yes]/X[J10 no ]/X[K11 no ]/X[L12 
> no ]/X[M13 yes]/X[N14 yes]/"
>   expect:  "1:0:[A01 no ]/2:10:[B02 no ]/?0/X[C03 yes]/X[D04 yes]/X[E05 yes]/X[F06 
> no ]/X[G07 no ]/X[H08 yes]/X[I09 yes]/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13 yes]/X[N14 
> yes]/"
>     FAIL

In tests 11 and 27, Grep is correct in outputting a context line after the 
requested number of selected lines.  This is explained in the manual under 
"--max-count": see below.

>  Test #28:  { ../src/grep -F -n -b -m 2 -v -C 1 -o yes; echo "?$?"; sed 
> 's!^!X!'; }
>   output:  "3-25-yes/?0/X[C03 yes]/X[D04 yes]/X[E05 yes]/X[F06 no ]/X[G07 no ]/X[H08 
> yes]/X[I09 yes]/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13 yes]/X[N14 yes]/"
>   expect:  "?0/X[C03 yes]/X[D04 yes]/X[E05 yes]/X[F06 no ]/X[G07 no ]/X[H08 
> yes]/X[I09 yes]/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13 yes]/X[N14 yes]/"
>     FAIL

We'll discuss the interaction between "-v" and "-o" separately.

> They all seem to have this in common: with -m,
> a rejected line from stdin (regular file) is
> both printed as context and left as unread (or
> seeked-back) input for the next process to read:
>
> ========================================
> 10-90-[J10 no ]/?0/X[J10 no ]/
> 3-20-[C03 yes]/?0/X[C03 yes]/
> 3-25-yes/?0/X[C03 yes]/
> ========================================
>
> This opportunity for repeated processing appears
> to go against the principles behind the design of
> -m, as detailed in grep's TeXinfo documentation.

I disagree: I think the documentation says this is done on purpose:

> `--max-count=NUM'
>      Stop reading a file after NUM matching lines.  If the input is
>      standard input from a regular file, and NUM matching lines are
>      output, `grep' ensures that the standard input is positioned to
>      just after the last matching line before exiting, regardless of the
>      presence of trailing context lines.

"Regardless of the presence..." is perhaps a little unclear, but from 
observation of the behaviour and from this later text:

>      When `grep' stops after NUM matching lines, it outputs any
>      trailing context lines.

... I find it means "even when trailing context lines have been printed."

The kind of use I can envisage for this is, with --max-count=1, to invoke 
Grep[*] repeatedly on the same input stream, displaying a match (in context) 
and asking for some user input in response before moving on to the next match. 
 The "example" given in the "info" document is rather poor and doesn't make 
its purpose clear but it does this kind of thing.

* "Grep": the program/utility/functionality/software, one of whose executables 
is normally called "grep" - sorry if my capitalisation is annoying anybody.


> Compare this to the failing "4/..." tests in tests/foad1.sh.
>
> ========================================
> Testing:  ../src/grep ^4 -m1 -A99
>   input:  "4/40/"
>   output: "4/"
>   expect: "4/40/"
> FAIL
[...]
> They deal with lines that would have
> been selected but that are beyond the -m
> specification.  The apparent expectation, by the
> design of those tests, is that a selectable but
> beyond -m line should be turned into a context
> line because of a positive -A specification.
> Is this desirable?  Should these tests be
> redesigned to match the current output in this
> case, which can be argued to be correct?

The description of "--max-count" in the "info" document says explicitly that 
such lines (that would be selected if given the chance) are NOT context lines.

That means those "4/..." tests are wrong.  They were added to test for seg 
faults; the output didn't matter for that purpose so I probably just put what I 
thought the correct output should be, without knowing about that definition of 
context lines.  (The "man" page doesn't say that and nor do the descriptions of 
context lines.  Grrr.)

I see that the "--max-count" feature was only added in March 2000, which is 
recent in terms of Grep's history.  Thus we can expect to have to examine and 
specify the edge-case behaviours for "--max-count".

- Julian