Re: Recent updates to tree-sitter branch

[Ihor Radchenko]

Yuan Fu <casouri@gmail.com> writes:

>>> Also bear in mind that the override flag can only be applied to the whole 
>>> query, rather than individual captured nodes.
>> 
>> How does it change anything? I may be misunderstanding something---can
>> you provide some illustrative example clarifying whole query vs.
>> individual notes?
>
> What I meant is that, for font-lock-keywords, one can set override flag for 
> each individual match:
>
> (string-regex font-lock-string-face t)
> (function-name-regexp font-lock-function-name-face nil)
> (class-name-regexp font-lock-type-face t)
> ...
>
> But for tree-sitter, a query contains many matches and the flag is set for 
> the query. So if I want to use different override flag for different matches, 
> I need to split them into two queries:
>
> (treesit-font-lock-rules
>  :language 'python
>  :override 'append
>  '((string) @python--treesit-fontify-string
>    ((string) @font-lock-doc-face
>     (:match "^\"\"\"" @font-lock-doc-face))
>    (interpolation (identifier) @font-lock-variable-name-face))
>
>  :language 'python
>  :override nil
>  '((function_definition
>     name: (identifier) @font-lock-function-name-face)
>
>    (class_definition
>     name: (identifier) @font-lock-type-face)
>
>    ;; Comment and string.
>    (comment) @font-lock-comment-face))

> That means if we use override=nil as default, it is very likely that users 
> need to explicitly set override to t for the whole query, or split the query 
> into separate parts. Nothing serious, but it seems less convenient.

What about allowing (@python--treesit-fontify-string 'append) to specify
the override?

> A real use-case for override is how I fontified Python strings above. I have 
> three matches for (1) all strings (2) docstrings (3) variable names in string 
> interpolations. IMO it’s intuitive and convenient for later more specific 
> matches to override earlier more general matches.

The current convention in font-lock-keywords is exactly opposite -
earlier matches are more specific, and they are later not replaced by
later more general matches.

Also, for reference, I am currently developing parser-based
fontification for Org.

I am using a somewhat different approach (closer to font-lock-keywords):

((drawer property-drawer) ;; <- match node types
  (:begin-marker 'org-drawer t) ;; <- apply fontification to :begin-marker 
field inside 
  (:end-marker 'org-drawer t)) ;;  <- ...                    :end-marker ....
((headline inlinetask)
  (:title-line
   (if (org-element-match-property :archivedp) ;; <- Elisp matching of the node 
properties
       'org-archived
     (pcase (org-element-match-property :todo-type) ;; <- ....
       (`todo (when org-fontify-todo-headline 'org-headline-todo))
       (`done (when org-fontify-done-headline 'org-headline-done))
       (_ nil)))
   t)) ;; <- override
 ((bold italic underline verbatim code strike-through)
   (:full-no-blank '(face nil org-emphasis t))) ;; <- fontify contents of the 
matched node
   
Also, see 
https://github.com/yantar92/org/blob/feature/org-font-lock-element/lisp/org-font-lock.el#L574

>From my experience re-implementing the vanilla fontification,
fontification order is important and may create subtle issues when not
designed carefully.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>