[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: A read-based grep-like for symbols (el-search?) (was Do shorthands b

From: Dmitry Gutov
Subject: Re: A read-based grep-like for symbols (el-search?) (was Do shorthands break basic tooling (tags, grep, etc)? (was Re: Shorthands have landed on master))
Date: Sat, 2 Oct 2021 04:14:23 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0

On 01.10.2021 19:41, João Távora wrote:
João Távora <joaotavora@gmail.com> writes:

João Távora <joaotavora@gmail.com> writes:

(benchmark 1 '(dolist (dir load-path)
                  (when (file-exists-p dir)
                    (let ((files (directory-files dir t "\\.el\\'")))
                      (dolist (file files)
                        (unless (file-directory-p file)
                            (insert-file file)
                            (read-from-string (buffer-string)))))))))

Reports 2.5 seconds here.

Are these Emacs's files?

Nevermind, just saw it's load-path.  Which makes sense.

So I have to ask: is this an 'Emacs -Q' load-path?

Anyway, the good news is that:

  * avoiding read-from-string, and using read isntead
  * using a single buffer
  * using insert-file-contents (insert-file is for interactive use)
  * using the Emacs -Q load-path

speeds your snippet down from 2 to 0.2 seconds.

The bad news is that your snippet was only reading the first form in a
file, so when you read the whole file it takes 2.8 seconds on my corei7
machine to read the whole Emacs sources.


Note that the above code uses very little Lisp and mostly calls into C. Any kind of deeper analysis based on reading the forms would need to have more processing implemented in Lisp around it. Which is often the slow part.

So performance wise, it's not bad but can probably be improved.  I don't think
it's gonna be stupidly hard because:

* this is one of those embarassingly parallel problems.  We could just spawn
   spawn `invocation-name' to use SMP

If analysis is implemented in Lisp, the concurrency will be limited by that part. Though I guess you could launch multiple Emacs instances and then combine their results somehow.

* designing a caching strategy seems fairly easy here.

Probably doable, yes, though both harder than the approach I described, and almost certainly slower even with cache (which needs regular invalidation). And the first search in the session (without cache to fall back on) will likely hurt the most.

Further, the code above only looks inside load-path one level deep. But we'd usually need to scan each directory recursively (there are packages like cedet with nested file structure).

Anyway, good luck. I just hope you don't end up with etags-like approach where the user will need to built and rebuild the index manually every time they want it to be fresh.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]