bug#60237: 30.0.50; tree sitter core dumps when I edebug view a node

[Yuan Fu]

Mickey Petersen <mickey@masteringemacs.org> writes:

> Yuan Fu <casouri@gmail.com> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Mickey Petersen <mickey@masteringemacs.org>
>>>> Date: Wed, 21 Dec 2022 12:24:34 +0000
>>>
>>> Yuan, can you look into this?  The crash is in tree-sitter, so maybe
>>> it isn't our bug, but I'd like to be sure.  And even if it is a
>>> tree-sitter bug, maybe we can work around it to prevent Emacs from
>>> crashing?
>>
>> Absolutely.
>>
>>>> Happens in emacs -Q (after loading some simple elisp code that uses 
>>>> treesit.el) and consistently and repeatedly.
>>>>
>>>>
>>>> Here's the elisp. When I edebug it I can step and view all the
>>>> variables and expressions I like. The `combobulate-' functions are
>>>> widely used in the library and pose no issues anywhere else and do
>>>> nothing more than fetch nodes via tree sitter. It is only this bit of
>>>> code that blows up, and then only when invoked inside a python
>>>> string.
>>
>> It would be nice if you can make a reproduce recipe. Judging from the
>> backtrace, you can probably trigger it by printing the node with print
>> or princ.  And does it trigger on all python strings? Or some specific
>> string in some specific python source?
>>
>
> This issue seems entirely related to `M-x treesit-explore-mode` (and
> possibly the inspect variant also) though it is hard to reproduce
> reliably. I get either crashes or hangs, depending on whether I have
> edebug on or not.
>
> Thrown errors seem to be the common denominator?

I’m stumbled on a reliably way to trigger a crash, of possibly the same cause as
this one, by enabling the profiler and M-x garbage-collect in a
tree-sitter mode on Mac. I tried to reproduce this on Linux but with no
success.

I was also able to trigger infinite loop by the same recipe on time, but I
didn’t run that session under lldb. Anyway, we can focus on the crash
first.

Below’s the backtrace. Eli, could you see anything from this? I have the
lldb session live so let me know if you want to see anything.
Unfortunately I can’t get gdb to work on Mac.

I suspect there is some stupid mistake that I made concerning gcing
tree-sitter objects. Could you see anything suspicious from the
following description:

A Lisp_TS_Parser contains a TSParser and a TSTree, which are freed when
the Lisp_TS_Parser is collected. A Lisp_TS_Node references the parser
from which it is created, so that a Lisp_TS_Parser is only collected
when no live node references it, because the Lisp_TS_Node references the
TSTree stored in the Lisp_TS_Parser.


* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION 
(code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x0000000100250f3d emacs`ASIZE(array=0x00000001a1889245) at 
lisp.h:1768:3
   1765 ASIZE (Lisp_Object array)
   1766 {
   1767   ptrdiff_t size = XVECTOR (array)->header.size;
-> 1768   eassume (0 <= size);
   1769   return size;
   1770 }
   1771
Target 0: (emacs) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION 
(code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x0000000100250f3d emacs`ASIZE(array=0x00000001a1889245) at 
lisp.h:1768:3
    frame #1: 0x0000000100250e5e emacs`get_backtrace(array=0x00000001a1889245) 
at eval.c:4193:28
    frame #2: 0x00000001003001ce emacs`record_backtrace(log=0x00000001a1887d68, 
count=64) at profiler.c:162:3
    frame #3: 0x000000010030016d emacs`malloc_probe(size=64) at profiler.c:509:3
    frame #4: 0x0000000100204e6d emacs`xmalloc(size=64) at alloc.c:760:3
    frame #5: 0x0000000100e6c0c9 libtree-sitter.0.dylib`ts_subtree_release + 158
    frame #6: 0x0000000100e6f004 libtree-sitter.0.dylib`ts_tree_delete + 44
    frame #7: 0x0000000100307379 
emacs`treesit_delete_parser(lisp_parser=0x00000001a2c0f0e0) at treesit.c:1182:3
    frame #8: 0x0000000100212c1b 
emacs`cleanup_vector(vector=0x00000001a2c0f0e0) at alloc.c:3179:5
    frame #9: 0x00000001002124c9 emacs`sweep_vectors at alloc.c:3254:5
    frame #10: 0x000000010020c777 emacs`gc_sweep at alloc.c:7430:3
    frame #11: 0x000000010020bb67 emacs`garbage_collect at alloc.c:6262:3
    frame #12: 0x000000010020b706 emacs`maybe_garbage_collect at alloc.c:6107:5
    frame #13: 0x00000001002b4bea emacs`maybe_gc at lisp.h:5591:5
    frame #14: 0x00000001002afcaa emacs`exec_byte_code(fun=0x0000000107557b85, 
args_template=256, nargs=1, args=0x000000010809e338) at bytecode.c:782:6
    frame #15: 0x0000000100251e77 
emacs`fetch_and_exec_byte_code(fun=0x00000001a1396be5, args_template=514, 
nargs=2, args=0x00007ff7bfefc628) at eval.c:3081:10
    frame #16: 0x000000010024e7c1 emacs`funcall_lambda(fun=0x00000001a1396be5, 
nargs=2, arg_vector=0x00007ff7bfefc628) at eval.c:3153:9
    frame #17: 0x000000010024e0a7 emacs`funcall_general(fun=0x00000001a1396be5, 
numargs=2, args=0x00007ff7bfefc628) at eval.c:2945:12
    frame #18: 0x00000001002494b4 emacs`Ffuncall(nargs=3, 
args=0x00007ff7bfefc620) at eval.c:2995:21
    frame #19: 0x000000010024d61c emacs`Fapply(nargs=2, 
args=0x00007ff7bfefc738) at eval.c:2666:24
    frame #20: 0x0000000100245752 emacs`apply1(fn=0x00000000a0a4d740, 
arg=0x00000001a38c59b3) at eval.c:2882:43
    frame #21: 0x00000001002cb671 
emacs`read_process_output_call(fun_and_args=0x00000001a38c59c3) at 
process.c:6070:10
    frame #22: 0x000000010024a493 
emacs`internal_condition_case_1(bfun=(emacs`read_process_output_call at 
process.c:6069), arg=0x00000001a38c59c3, handlers=0x0000000000000090, 
hfun=(emacs`read_process_output_error_handler at process.c:6075)) at 
eval.c:1498:25
    frame #23: 0x00000001002cb5c3 
emacs`read_and_dispose_of_process_output(p=0x00000001a1448440, 
chars="Content-Length: 
1184\r\n\r\n{\"jsonrpc\":\"2.0\",\"method\":\"textDocument/publishDiagnostics\",\"params\":{\"uri\":\"file:///Users/yuan/t/js/test.ts\",\"diagnostics\":[{\"range\":{\"start\":{\"line\":4,\"character\":11},\"end\":{\"line\":4,\"character\":14}},\"message\":\"Property
 'get' does not exist on type 
'Document'.\",\"severity\":1,\"code\":2339,\"source\":\"typescript\",\"tags\":[]},{\"range\":{\"start\":{\"line\":0,\"character\":14},\"end\":{\"line\":0,\"character\":15}},\"message\":\"Parameter
 'a' implicitly has an 'any' type, but a better type may be inferred from 
usage.\",\"severity\":4,\"code\":7044,\"source\":\"typescript\",\"tags\":[]},{\"range\":{\"start\":{\"line\":0,\"character\":14},\"end\":{\"line\":0,\"character\":15}},\"message\":\"'a'
 is declared but its value is never 
read.\",\"severity\":4,\"code\":6133,\"source\":\"typescript\",\"tags\":[1]},{\"range\":{\"start\":{\"line\":0,\"character\":17},\"end\":{\"line\":0,\"character\":18}},\"message\":\"Parameter
 'b' implicitly has an 'any' type, but a better type may be inferred from 
usage.\",\"severity\":4,\"code\":7044,\"source\":\"typescript\",\"tags\":[]},{\"range\":{\""...,
 nbytes=1208, coding=0x00000001577aed10) at process.c:6294:5
    frame #24: 0x00000001002c46df 
emacs`read_process_output(proc=0x00000001a1448445, channel=26) at 
process.c:6204:3
    frame #25: 0x00000001002c3097 
emacs`wait_reading_process_output(time_limit=5, nsecs=0, read_kbd=-1, 
do_display=true, wait_for_cell=0x0000000000000000, 
wait_proc=0x0000000000000000, just_wait_proc=0) at process.c:5888:16
    frame #26: 0x000000010000c881 emacs`sit_for(timeout=0x0000000000000016, 
reading=true, display_option=1) at dispnew.c:6256:7
    frame #27: 0x0000000100166ffd emacs`read_char(commandflag=1, 
map=0x0000000193762fb3, prev_event=0x0000000000000000, 
used_mouse_menu=0x00007ff7bfefeb1f, end_time=0x0000000000000000) at 
keyboard.c:2872:11
    frame #28: 0x0000000100162e18 
emacs`read_key_sequence(keybuf=0x00007ff7bfefee40, prompt=0x0000000000000000, 
dont_downcase_last=false, can_return_switch_frame=true, 
fix_current_buffer=true, prevent_redisplay=false) at keyboard.c:10074:12
    frame #29: 0x00000001001611c0 emacs`command_loop_1 at keyboard.c:1375:15
    frame #30: 0x000000010024a3c8 
emacs`internal_condition_case(bfun=(emacs`command_loop_1 at keyboard.c:1269), 
handlers=0x0000000000000090, hfun=(emacs`cmd_error at keyboard.c:927)) at 
eval.c:1474:25
    frame #31: 0x0000000100160c63 
emacs`command_loop_2(handlers=0x0000000000000090) at keyboard.c:1124:11
    frame #32: 0x0000000100249b53 emacs`internal_catch(tag=0x000000000000f3f0, 
func=(emacs`command_loop_2 at keyboard.c:1120), arg=0x0000000000000090) at 
eval.c:1197:25
    frame #33: 0x000000010015ffda emacs`command_loop at keyboard.c:1102:2
    frame #34: 0x000000010015fddf emacs`recursive_edit_1 at keyboard.c:711:9
    frame #35: 0x0000000100160352 emacs`Frecursive_edit at keyboard.c:794:3
    frame #36: 0x000000010015d177 emacs`main(argc=1, argv=0x00007ff7bfeff6a8) 
at emacs.c:2529:3
    frame #37: 0x00007ff81a314310 dyld`start + 2432
(lldb) up 4
frame #4: 0x0000000100204e6d emacs`xmalloc(size=64) at alloc.c:760:3
   757
   758    if (!val)
   759      memory_full (size);
-> 760    MALLOC_PROBE (size);
   761    return val;
   762  }
   763
(lldb) list
   764  /* Like the above, but zeroes out the memory just allocated.  */
   765
   766  void *
   767  xzalloc (size_t size)
   768  {
   769    void *val;
   770
(lldb) up
frame #5: 0x0000000100e6c0c9 libtree-sitter.0.dylib`ts_subtree_release + 158
libtree-sitter.0.dylib`ts_subtree_release:
->  0x100e6c0c9 <+158>: movq   -0x30(%rbp), %rdi
    0x100e6c0cd <+162>: movq   %rax, 0x10(%rdi)
    0x100e6c0d1 <+166>: movl   %r15d, 0x1c(%rdi)
    0x100e6c0d5 <+170>: movl   0x18(%rdi), %eax
(lldb) down
frame #4: 0x0000000100204e6d emacs`xmalloc(size=64) at alloc.c:760:3
   757
   758    if (!val)
   759      memory_full (size);
-> 760    MALLOC_PROBE (size);
   761    return val;
   762  }
   763
(lldb) down
frame #3: 0x000000010030016d emacs`malloc_probe(size=64) at profiler.c:509:3
   506  malloc_probe (size_t size)
   507  {
   508    eassert (HASH_TABLE_P (memory_log));
-> 509    record_backtrace (XHASH_TABLE (memory_log), min (size, 
MOST_POSITIVE_FIXNUM));
   510  }
   511
   512  DEFUN ("function-equal", Ffunction_equal, Sfunction_equal, 2, 2, 0,
(lldb) down
frame #2: 0x00000001003001ce emacs`record_backtrace(log=0x00000001a1887d68, 
count=64) at profiler.c:162:3
   159    /* Get a "working memory" vector.  */
   160    Lisp_Object backtrace = HASH_VALUE (log, index);
   161    eassert (BASE_EQ (Qunbound, HASH_KEY (log, index)));
-> 162    get_backtrace (backtrace);
   163
   164    { /* We basically do a `gethash+puthash' here, except that we have to 
be
   165         careful to avoid memory allocation since we're in a signal
(lldb) down
frame #1: 0x0000000100250e5e emacs`get_backtrace(array=0x00000001a1889245) at 
eval.c:4193:28
   4190 get_backtrace (Lisp_Object array)
   4191 {
   4192   union specbinding *pdl = backtrace_next (backtrace_top ());
-> 4193   ptrdiff_t i = 0, asize = ASIZE (array);
   4194
   4195   /* Copy the backtrace contents into working memory.  */
   4196   for (; i < asize; i++)
(lldb) down
frame #0: 0x0000000100250f3d emacs`ASIZE(array=0x00000001a1889245) at 
lisp.h:1768:3
   1765 ASIZE (Lisp_Object array)
   1766 {
   1767   ptrdiff_t size = XVECTOR (array)->header.size;
-> 1768   eassume (0 <= size);
   1769   return size;
   1770 }
   1771
(lldb) p size
(ptrdiff_t) $0 = -9223372036854775792
(lldb) p XVECTOR(array)
(Lisp_Vector *) $1 = 0x00000001a1889240
(lldb) p XVECTOR(array)->header
(vectorlike_header) $2 = (size = -9223372036854775792)
(lldb) p *array
error: expression failed to parse:
error: <user expression 3>:1:1: incomplete type 'Lisp_X' where a complete type 
is required
*array
^
(lldb)