bug#14391: bug in grep -- ignoring GREP_OPTIONS or not acting on it...

[Bob Proulx]

Linda Walsh wrote:
> Bob Proulx wrote:
> > Basically:
> > 
> > I think you need --directories=skip in addition to --devices=skip.
> > Add -d skip to your option list.
> > 
> >   grep -D skip -d skip --binary-files=without-match --color=auto foo *

> Very good points -- especially about my broken options (I could
> swear they worked in the past)...must have gotten accidently deleted
> somewhere along in the past.

It probably did work in the past.  If you look through all of the
previous grep versions you will probably find at least one where that
combination worked.  I didn't mean to say otherwise.  I remember
grep's recursion behavior as having gone through several iterations of
changes.  It hasn't matured yet.  But hopefully the changes are
slowing down.  :-)

> > This is an example of why I think grep should never have added
> > directory recursion.  It needs to suck in all of 'find' before it will
> > be complete.  And that doesn't make sense because there is already
> > find available.
> > 
> >   find . -type f -exec grep foo {} +
> 
> But .. I also agree with the problems of grep possibly calling find
> inefficiently.  I would have thought it smarter to call find as a
> child, and read from it's pipe and call the file-search on large
> groups at a time.  Otherwise, if grep waited for all of find to
> complete, as you are suggesting, might it not run out of space to
> hold the paths of the files it is going to search?

Using the above with "{} +" is very efficient.  It will not exceed
ARG_MAX.  The "{} +" part of the find instructs find to:

  3.3.2 Multiple Files
  --------------------

  Sometimes you need to process files one at a time.  But usually this is
  not necessary, and, it is faster to run a command on as many files as
  possible at a time, rather than once per file.  Doing this saves on the
  time it takes to start up the command each time.

     The `-execdir' and `-exec' actions have variants that build command
  lines containing as many matched files as possible.

   -- Action: -execdir command {} +
       This works as for `-execdir command ;', except that the `{}' at
       the end of the command is expanded to a list of names of matching
       files.  This expansion is done in such a way as to avoid exceeding
       the maximum command line length available on the system.  Only one
       `{}' is allowed within the command, and it must appear at the end,
       immediately before the `+'.  A `+' appearing in any position other
       than immediately after `{}' is not considered to be special (that
       is, it does not terminate the command).

   -- Action: -exec command {} +
       This insecure variant of the `-execdir' action is specified by
       POSIX.  The main difference is that the command is executed in the
       directory from which `find' was invoked, meaning that `{}' is
       expanded to a relative path starting with the name of one of the
       starting directories, rather than just the basename of the matched
       file.

The old traditional way was "\;" (needed to be quoted to protect it
from the shell) which invoked the command once for each argument.  The
new standard way is "+" which bundles up as many arguments as
possible, will not exceed ARG_MAX, and therefore is quite efficient.
Also it doesn't need to be special quoting making it more convenient.

Using 'find . -print0 | grep --files0-from=- ...', if implemented,
would also be efficient.  But not implemented AFAIK.

I think the "insecure" warning is too agressive and overstating the point.

Bob