bug#50733: 28.0.1; project-find-regexp can block Emacs for a long time

[Gregory Heytings]

> > Hint: read again, and look closer.  gid returns some matches in 
> > comments, some not.  For no apparent reason.
>
> Of course, there is a reason, you just don't understand it

Who told you that I'm talking about myself?  FWIW, I understand the 
reason, but it's a completely non-apparent one, that isn't documented 
anywhere.  And it's not one that would make sense to most users.

> (and generally think about gid as if it were a general-purpose 
> text-search utility).

Who told you that I'm talking about myself?  I do not think of gid as a 
general-purpose text-search utility, but what most users want (and what 
project-find-regexp needs) is an efficient, easy to use and understand, 
predictable general-purpose text-search utility.  ripgrep is one such 
utility, idutils is not.  And, once again, ripgrep is, from a user point 
of view, as fast as or faster than idutils, and does not require any 
plumbing behind the scenes.

Out of curiosity, because of your "it doesn't scale" remark, I just 
compared the efficiency of ripgrep and idutils on the latest Linux kernel 
tarball (1.4 GB in 78464 files):

mkid takes 31 seconds

rg O_CREAT takes 0.18 seconds
gid O_CREAT takes 0.02 seconds
rg O.?CREAT takes 0.18 seconds
gid O.?CREAT takes 0.93 seconds
rg O.*CREAT takes 0.19 seconds
gid O.*CREAT takes 1.73 seconds

Isn't idutils the one that doesn't scale?

The only case in which idutils is faster (if one does not take the time 
that was spent to build the database into account, and if one considers 
that it's okay to ignore some matches in comments) is a plain identifier; 
from a user viewpoint getting an answer in 0.2 seconds on such a big code 
base is as good as getting an answer in 0.02 seconds.  It's slower, much 
slower in all other cases, whenever a regexp is used --- which is what 
project-find-regexp is all about.