bug#60127: treesit-end-of-defun: possible bug with clojure grammar

[Danny Freeman]

Hello,

I have been running into what I believe is a bug with
treesit-end-of-defun while developing clojure-ts-mode. There are some
positions in clojure buffers where treesit-end-of-defun will jump to the
end of the buffer.

Steps to reproduce:

Checkout the emacs-29 branch. I have built mine with commit
b01d0246d71a7a3fd92b2864a3c0c0bc9367ee0b and tree-sitter version 0.20.7

Clone my clojure-mode fork:
https://github.com/dannyfreeman/clojure-mode

check out the `end-of-defun-bug` branch (master works too, but I added
a helpful debug message in this branch)

Clone tree-sitter-module repo:
https://github.com/casouri/tree-sitter-module

Apply the patch `tree-sitter-module-clojure-support.patch` located in
clojure-mode repo to the casouri/tree-sitter-module repo.

Build the treesitter parser for clojure with

$ ./build clojure

Start emacs

$ emacs -q

Evaluate the following, replacing the paths with what is relevant to
your system

```
(progn
  (setq treesit-extra-load-path '("~/path/to/tree-sitter-module/dist"))
  (add-to-list 'load-path "~/path/to/clojure-mode/")
  (find-file "~/path/to/clojure-mode/clojure-ts-mode.el")
  (eval-buffer)
  (find-file "~/path/to/clojure-mode/test_end_of_defun.clj"))
```

Once you have opened test_end_of_defun.clj you can see the problem by
pressing

M-<
C-M-e
C-M-e

You should see the cursor move from

```
|(def w 1)

(def x 2)

(def y 3)

(def skip-to-here? 4)
```

to
```
(def w 1)
|
(def x 2)

(def y 3)

(def skip-to-here? 4)
```

then it unexpectedly jumps to the end of the buffer
```
(def w 1)

(def x 2)

(def y 3)

(def skip-to-here? 4)
|
```

The message buffer shows that when the point is between the forms

```
(def w 1)
|
(def x 2)
```

that `treesit-end-of-defun` thinks the current node is
`(def skip-to-here? 4)`, which does not seem right.

When the cursor is at other points in this buffer, this problem doesn't
seem to occur. For instance, between the (def x 2) and (def y 3) forms,
this doesn't happen.

I can also see this happening in the `test.clj` file of the clojure-mode
repo in a different spot but there is a lot more going on in that file.
(hit C-M-e in it until the problem occurs if you are curious).

I have the following vars relevant vars set in clojure-ts-mode.
Different combinations of them yield the same results.

```
(setq-local treesit-defun-prefer-top-level t
            treesit-defun-tactic 'top-level
            treesit-defun-type-regexp (cons (rx (or "list_lit" "vec_lit" 
"map_lit"))
                                            (lambda (node)
                                              (message "Node: %s" 
(treesit-node-text node t))
                                              t)))
```

The clojure code in question produces an error free parse tree. I can
see it with `treesit-explore-mode` and by running the file through
`tree-sitter parse`, which gives the following parse tree:

```
(source [0, 0] - [7, 0]
  (list_lit [0, 0] - [0, 9]
    value: (sym_lit [0, 1] - [0, 4]
      name: (sym_name [0, 1] - [0, 4]))
    value: (sym_lit [0, 5] - [0, 6]
      name: (sym_name [0, 5] - [0, 6]))
    value: (num_lit [0, 7] - [0, 8]))
  (list_lit [2, 0] - [2, 9]
    value: (sym_lit [2, 1] - [2, 4]
      name: (sym_name [2, 1] - [2, 4]))
    value: (sym_lit [2, 5] - [2, 6]
      name: (sym_name [2, 5] - [2, 6]))
    value: (num_lit [2, 7] - [2, 8]))
  (list_lit [4, 0] - [4, 9]
    value: (sym_lit [4, 1] - [4, 4]
      name: (sym_name [4, 1] - [4, 4]))
    value: (sym_lit [4, 5] - [4, 6]
      name: (sym_name [4, 5] - [4, 6]))
    value: (num_lit [4, 7] - [4, 8]))
  (list_lit [6, 0] - [6, 21]
    value: (sym_lit [6, 1] - [6, 4]
      name: (sym_name [6, 1] - [6, 4]))
    value: (sym_lit [6, 5] - [6, 18]
      name: (sym_name [6, 5] - [6, 18]))
    value: (num_lit [6, 19] - [6, 20])))
```

Any help or advice here is appreciated.

Thank you,
-- 
Danny Freeman