emacs-tree-sitter and font-lock (was: emacs rendering comparisson between emacs23 and emacs26.3)

[Stefan Monnier]

>> I think it's just a reflection of the fact that author doesn't know
>> enough about jit-lock and font-lock.
> Indeed, that font-lock rule can be a function is not very well-known.

Although it seems fairly clearly documented.
Not sure how we could solve this.

>> I sent him some patches that should help.
> That's mighty good of you.
> I was curious and wanted to take a look, though, but couldn't find them
> neither among pull requests, nor the issue comments. :-(

I don't how to participate in a Github pull request with
Free Software.  IIUC there are some Elisp packages for that, but
I haven't investigated how that would work.
So I sent the patches via email.  I put the main one after my sig.


        Stefan


diff --git a/lisp/tree-sitter-highlight.el b/lisp/tree-sitter-highlight.el
index 53ecb32..3246bd6 100644
--- a/lisp/tree-sitter-highlight.el
+++ b/lisp/tree-sitter-highlight.el
@@ -167,7 +167,6 @@ to faces.  Each function takes no arguments."
 (defvar-local tree-sitter-highlight--injections nil)
 (defvar-local tree-sitter-highlight--injections-query nil)
 (defvar-local tree-sitter-highlight--jit-function nil)
-(defvar-local tree-sitter-highlight--orig-scroll-functions nil)
 (defvar-local tree-sitter-highlight--query nil)
 (defvar-local tree-sitter-highlight--query-cursor nil)
 
@@ -240,19 +239,11 @@ to faces.  Each function takes no arguments."
 
 (defun tree-sitter-highlight--jit (old-tree)
   (when old-tree
-    (let ((changes (ts-changed-ranges old-tree tree-sitter-tree))
-           (wstart (window-start))
-           (wend (window-end)))
-      ;; TODO: Remember what we've highlighted, similar to how font-lock does 
it.
-      ;;       Already highlighted regions shouldn't be re-highlighted.
-
-      ;; Find changes that are within the current window
-      (mapc #'(lambda (range)
-                (let ((start (aref range 0))
-                       (end (aref range 1)))
-                  ;; TODO: Improve this
-                  (tree-sitter-highlight--highlight (max wstart start) (min 
wend end))))
-        changes))))
+    (mapc #'(lambda (range)
+              (let ((start (aref range 0))
+                    (end (aref range 1)))
+                (font-lock-flush start end)))
+          (ts-changed-ranges old-tree tree-sitter-tree))))
 
 (defun tree-sitter-highlight--highlight (start end)
   (ts--save-context
@@ -282,9 +273,6 @@ to faces.  Each function takes no arguments."
 ;;               (ts-byte-from-position start)
 ;;               (ts-byte-from-position end)))))
 
-(defun tree-sitter-highlight--highlight-window (_window start)
-  (tree-sitter-highlight--highlight start (window-end nil t)))
-
 (defun tree-sitter-highlight--enable ()
   "Enable `tree-sitter-highlight' in this buffer."
   (run-hooks  'tree-sitter-highlight-setup-functions)
@@ -301,10 +289,8 @@ to faces.  Each function takes no arguments."
       tree-sitter-highlight--injections-query (cadr x)))
   (setq tree-sitter-highlight--capture-names (ts-query-capture-names 
tree-sitter-highlight--query))
   (setq tree-sitter-highlight--query-cursor (ts-make-query-cursor))
-  (make-variable-buffer-local 'window-scroll-functions)
-  (setq tree-sitter-highlight--orig-scroll-functions window-scroll-functions)
-  (setq window-scroll-functions (cons 
#'tree-sitter-highlight--highlight-window window-scroll-functions))
-  (tree-sitter-highlight--highlight-window nil (window-start))
+  (add-function :override (local 'font-lock-fontify-region-function)
+                #'tree-sitter-highlight--highlight)
   (add-hook 'tree-sitter-after-change-functions #'tree-sitter-highlight--jit 
nil t)
   )
 
@@ -314,7 +300,8 @@ to faces.  Each function takes no arguments."
     (remove-text-properties (point-min)
       (point-max)
       '(face nil)))
-  (setq window-scroll-functions tree-sitter-highlight--orig-scroll-functions)
+  (remove-function (local 'font-lock-fontify-region-function)
+                   #'tree-sitter-highlight--highlight)
   (remove-hook 'tree-sitter-after-change-functions 
#'tree-sitter-highlight--jit t))
 
 (define-minor-mode tree-sitter-highlight-mode