Re: Tree-sitter api

[Tuấn-Anh Nguyễn]

On Mon, Sep 6, 2021 at 3:19 AM Yuan Fu <casouri@gmail.com> wrote:
> Dynamic modules comes with nice things, for example Emacs looks for them 
> automatically in load-path; Emacs reports errors with it has problem loading 
> one; On the other hand, dynamic modules don’t come with much complications. 
> Yes, you need additional emacs-modules.h and tree-sitter-<lang>.c to build 
> it, but that’s about it.

See my other discussion with Eli. We want to rely on the distro to provide the
binaries and the `tree-sitter` CLI program, and to be able to use shared libs
from other sources as well (like self-built). They are not going to be Emacs
dynamic modules.

> I can package some additional information with the module; I could maybe 
> distribute them through ordinary package.el facility, etc etc.

Neither of these requires it to be a module at all. (Also note that package.el
isn't able to handle platform-specific files at the moment.)

> If I load the shared library directly, I need to reinvent the wheels for 
> loading, error reporting, searching in load-path, and others.

The non-module-specific part of loading is provided by `dynlib.h`. There's no
wheel to reinvent here. What error reporting do you mean? (You are going to need
additional checks for ABI compatibility anyway.) Searching a load path (not the
`load-path`) is not that complicated. What are the others?

> And I was hoping to distribute pre-built modules anyway, so if all went well, 
> ordinary users don’t need to compile the modules. WDYT?

It's good to provide that convenience, but it should not be at the expense of
not being able to use binaries from other sources, or to build the binaries on
their own. The `tree-sitter-langs` package already enables both of these. It
provides both pre-built binaries and functions for users to compile on their
own. And it does so without putting language definitions in dynamic modules.

> P.S. what do you mean by “load path pollution”?

I meant to say load path collision, but since you use `tree-sitter-{lang}` for
the module name, that's less of a problem. Load path pollution is these names
showing up when the user enumerates entries on the load path trying to go to the
source of a Lisp library. That's annoying, but bearable.

> P.P.S. My impression is that other applications distribute language 
> definitions by themselves, and it is not common for distort to package 
> language definitions, is that correct?

I don't understand this. Can you rephrase it?

All in all, you are severely underestimating the amount of complexity and wheels
you will have to reinvent in other places compared to the amount of code you
don't have to write by requiring language definitions to be in dynamic modules.
(It's less than 100, most of which is docstrings and comments.)

--
Tuấn-Anh Nguyễn
Software Engineer