Re: Call for volunteers: add tree-sitter support to major modes

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, I will.
>
> It isn't our business to develop language parsers, and we have no hope
> of keeping up with the technology developments in that area and with
> new languages being invented, well enough to offer reasonable and
> modern support in Emacs for editing programs in those languages.

It isn't quite our business to develop mini-gmp either, yet it's also
kept in-tree.

> It's the same reason why we depend on text shaping engines like
> HarfBuzz for rendering complex scripts and other typography-related
> features, on image libraries to display images, on GnuTLS to support
> TLS connections, etc. etc.

But editing text mostly does not involve editing complex shaped text,
and likewise, TLS connections and image display is mostly orthogonal to
the business of editing text.

> We cannot possibly be experts in all of those fields, we don't have
> the manpower for that.  In the case of support for editing programs we
> actually tried for many years, and the result is before our eyes --
> and it's unsatisfactory at best.  We should advance Emacs towards
> leveraging existing technologies, not try developing those
> technologies in-house, or replacing them by cheap hacks and kludges.

Then why is mini-gmp kept in-tree? Portable multiple precision
arithmetic is at least as complicated (if not more) than tree-sitter,
yet mini-gmp is a completely satisfactory solution for people who do not
have libgmp installed.

mini-tree-sitter would fill the same role: it would expose the same API
as tree-sitter, but be small enough for Emacs to include and use when
tree-sitter is not available.

> It is completely impractical to expect us to be able to develop this
> in-house.

Why?  The tree-sitter runtime library is only about 16,000 lines of C
source code and headers, which is less than xdisp.c alone, and only
slightly more than keyboard.c.  In addition, the path to the current
implementation of tree-sitter is well-trodden, so it can't be that hard
to walk down that path again.

Such an implementation would be able to make use of existing and new
tree-sitter grammars without changes.

> If tree-sitter becomes defunct (something that doesn't seem like it
> will happen any time soon)

Famous last words.

> , we will have to find a suitable replacement -- like we did with
> text-shaping engine, when the FLT library stopped being actively
> developed.  Dealing with these changes in the outside world is an
> important part of the job of the Emacs maintainers

Text shaping is, at least IME, immensely more complicated than
interactive language parsing.

> The present font-lock and indentation features based on our own code
> can remain in Emacs forever, if someone wants a safe fallback for when
> worse comes to worst.

That logic could also have been applied to bignums, yet there was no
objection to including mini-gmp.

> But these implementations are from my POV a dead end: they cannot be
> developed any further, and for some languages, like C++, are already
> all but failing to reasonably and efficiently support their complex
> grammars.  Using expertise of others is simply a logical step based on
> past experience; the decision to develop this in-house would be a step
> back and a disaster in the long run, IMNSHO.

I'm not disputing that fact, which is why I proposed to write an
*implementation* of tree-sitter for inclusion with Emacs, and not to
develop an equivalent in-house.