Re: A read-based grep-like for symbols (el-search?) (was Do shorthands break basic tooling (tags, grep, etc)? (was Re: Shorthands have landed on master))

[Dmitry Gutov]

On 01.10.2021 17:40, João Távora wrote:

> > Certain language designers intentionally limit the language's power due
> > to usability considerations, keeping in mind their audience.
>
> What languages, what evidence for this?  Anyway, many more limit the power
> due to performance considerations.  Counts as "usability"? I guess. IME
> language  audiences that are interested in performance usually don't
> care so much
> about ergonomics and vice versa.

Go would be one example. The reasoning lies largely in the field of 
usability. Their understanding of it, at least.

But being able to use certain common tools belongs to the same area.

> > Speaking of shorthands, if only the "local" part of every symbol's name
> > was something reliable (as is often the case in module/package systems
> > out there), we could still implement the search for references using
> > Grep fairly efficiently: you Grep across the files for the local name,
> > and then post-filter the references by looking at the end of the file.
>
> Yes, yes, that's basically what happens with Common Lisp, because things
> are separated by `:`.  And in many other languages as you say.  Unfortunately,
> it's not 100% clean in Elisp because it relies on convention, not syntax.

If we moved the declaration of the package prefix from the referring 
file (at the bottom) to the defining file, or just always said that the 
prefix must match the package name, this would make this possible. At 
the cost of some flexibility, of course, but I'm not sure who really 
needs that.

Taking the example from the manual, the clients would be able to write

;; elisp-shorthands: (("snu" . "some-nice-string-utils"))

but not

;; elisp-shorthands: (("sn" . "some-nice"))


and that doesn't sound like a terrible limitation.

> But
> possible, yes.  Would you like to work on that `thing-at-pt.el` front?

thing-at-pt? I'm not sure which particular task you are referring to.

> > If that approach is not feasible, we're limited to searching for the
> > instances of 'require' forms (when the symbol/function can be mapped to
> > a package name) and then searching inside every such file on the second
> > pass.
>
> That's not the approach I was thinking of, but I hope to present a working
> prototype soon, which is a better way to present ideas.

Looking forward to it.

> > read-ing the contents of every Lisp file is pretty expensive, in comparison.
>
> Have you benchmarked?  What exactly have you benchmarked? Just `read` or
> `read` + `parse-partial-sexp` (i.e. building the syntax-ppss cache)?
> Versus what?

I'm currently having a discussion where taking 2 seconds to search for a 
(very common) Lisp reference across Emacs code base is considered too 
slow, with all the Lisp overhead of parsing the results and constructing 
the list and rendering.

Simply doing

(benchmark 1 '(dolist (dir load-path)
                (when (file-exists-p dir)
                  (let ((files (directory-files dir t "\\.el\\'")))
                    (dolist (file files)
                      (unless (file-directory-p file)
                        (with-temp-buffer
                          (insert-file file)
                          (read-from-string (buffer-string)))))))))

Reports 2.5 seconds here.