Re: Tree-sitter api

[Yuan Fu]

> On Sep 5, 2021, at 5:03 PM, Tuấn-Anh Nguyễn <ubolonton@gmail.com> wrote:
> 
> On Mon, Sep 6, 2021 at 3:19 AM Yuan Fu <casouri@gmail.com> wrote:
>> Dynamic modules comes with nice things, for example Emacs looks for them 
>> automatically in load-path; Emacs reports errors with it has problem loading 
>> one; On the other hand, dynamic modules don’t come with much complications. 
>> Yes, you need additional emacs-modules.h and tree-sitter-<lang>.c to build 
>> it, but that’s about it.
> 
> See my other discussion with Eli. We want to rely on the distro to provide the
> binaries and the `tree-sitter` CLI program, and to be able to use shared libs
> from other sources as well (like self-built). They are not going to be Emacs
> dynamic modules.
> 
>> I can package some additional information with the module; I could maybe 
>> distribute them through ordinary package.el facility, etc etc.
> 
> Neither of these requires it to be a module at all. (Also note that package.el
> isn't able to handle platform-specific files at the moment.)
> 
>> If I load the shared library directly, I need to reinvent the wheels for 
>> loading, error reporting, searching in load-path, and others.
> 
> The non-module-specific part of loading is provided by `dynlib.h`. There's no
> wheel to reinvent here. What error reporting do you mean? (You are going to 
> need
> additional checks for ABI compatibility anyway.) Searching a load path (not 
> the
> `load-path`) is not that complicated. What are the others?
> 
>> And I was hoping to distribute pre-built modules anyway, so if all went 
>> well, ordinary users don’t need to compile the modules. WDYT?
> 
> It's good to provide that convenience, but it should not be at the expense of
> not being able to use binaries from other sources, or to build the binaries on
> their own. The `tree-sitter-langs` package already enables both of these. It
> provides both pre-built binaries and functions for users to compile on their
> own. And it does so without putting language definitions in dynamic modules.
> 
>> P.S. what do you mean by “load path pollution”?
> 
> I meant to say load path collision, but since you use `tree-sitter-{lang}` for
> the module name, that's less of a problem. Load path pollution is these names
> showing up when the user enumerates entries on the load path trying to go to 
> the
> source of a Lisp library. That's annoying, but bearable.
> 
>> P.P.S. My impression is that other applications distribute language 
>> definitions by themselves, and it is not common for distort to package 
>> language definitions, is that correct?
> 
> I don't understand this. Can you rephrase it?
> 
> All in all, you are severely underestimating the amount of complexity and 
> wheels
> you will have to reinvent in other places compared to the amount of code you
> don't have to write by requiring language definitions to be in dynamic 
> modules.
> (It's less than 100, most of which is docstrings and comments.)

I see your point. If no one else object, I’ll change the code to use shared 
libraries instead of dynamic modules. Thanks for the input :-)

Yuan