Re: Tree-sitter api

[Yuan Fu]

> On Sep 27, 2021, at 12:17 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> Currently, because font-lock.el uses functions and variables defined in
>> tree-sitter.el,
> 
> Why?  I don't see any reason why you'd need to change font-lock.el to
> add support for tree-sitter fontification.
> 

[Also in reply to Eli:]

Before tree-sitter, font-lock roughly consists of two passes, the syntactic 
pass (that uses the syntax table) and regex pass (that uses regex matching). I 
added a three pass, tree-sitter pass, because I want to add tree-sitter 
fontification on top of existing mechanisms, not replacing it. This way we can 
still fontify keywords. Simply replacing font-lock with tree-sitter font-lock 
would cause anything relying on the existing fontification facility stop to 
work if I turn on tree-sitter. For example, I use a package that fontify 
keywords like “TODO” and “FIXME”, it would be a shame if it stops working as 
soon as I turn on tree-sitter fontification. So it seemed natural to me to 
augment font-lock.el instead of putting stuff in tree-sitter.el. Now that I 
realized the dependency problem, I think I can move all the font-lock 
integration code to tree-sitter.el and leave font-lock.el untouched, but still 
maintain the augmentation nature. E.g., define a 
tree-sitter-fontify-region-function that first calls 
font-lock-fontify-region-function, then does tree-sitter fontification. And 
user can turn on tree-sitter font-lock with, say tree-sitter-font-lock-mode.

With that said, I still have one thing not too sure. What should tree-sitter.el 
do if libtree-sitter is not on the system, and tree-sitter.c is not included in 
Emacs? Should we simply not include tree-sitter.el? Is there existing build 
facility that can do that (exclude tree-sitter.el when libtree-sitter is not 
found on system)?

Yuan