bug#50733: 28.0.1; project-find-regexp can block Emacs for a long time

[Dmitry Gutov]

Hi Daniel,

On 22.09.2021 12:27, Daniel Martín via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> When trying to search in a big project with thousands of files using
> `project-find-regexp`, Emacs can synchronously block for a long time
> until the results are computed for display in an *xref* buffer.
>
> In contrast, the rgrep command and friends create a `grep-mode` buffer
> and asynchronously fill it with results as they are computed by the grep
> tool.  Emacs is not blocked and the user can start exploring matches
> right away.
>
> Is there a plan to eventually replace `grep-mode` with `xref-mode` in
> Emacs?

No solid plan to replace everything (a lot of people will object, for 
various reasons), but it's the direction I want to see. At least in a 
core set of IDE-ish commands.

> If so, I think that computing results asynchronously is the most
> important feature we'll lose.  When using `xref-mode` with an index
> (etags, for example), the blocking is not noticeable; but when using a
> grep tool the delay can be annoying if the repository is huge.

Non-blocking UI for Xref is possible, but it requires a fair amount of 
work, possibly some low-level one. Not in the cards for Emacs 28, at least.

But maybe we could improve things another way first.

IIRC you are using macOS. I received another report recently that 
find/grep based tooling, and project-find-regexp in particular, are 
pretty slow on that OS.

When you say "block for a long time", how long are we talking about?

For reference, when a do a search across Emacs sources checkout, and 
there are few matches, it returns in ~100ms (with warm cache and SSD 
disk, to be clear).

To try it, evaluate

  (benchmark 1 '(project-find-regexp "new-collection"))

And a similar search across the Linux kernel source tree takes ~1s.

The first thing to try is to either install the latest GNU Grep and make 
sure Emacs uses it, or install ripgrep and

  (setq xref-search-program 'ripgrep)

Please report if that helps and how much.

Another benchmark to try is

  (benchmark 1 '(project-files (project-current)))