[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Some issues with the tree-sitter branch
From: |
Yuan Fu |
Subject: |
Re: Some issues with the tree-sitter branch |
Date: |
Sun, 16 Oct 2022 22:48:35 -0700 |
> On Oct 16, 2022, at 10:37 PM, Po Lu <luangruo@yahoo.com> wrote:
>
> Yuan Fu <casouri@gmail.com> writes:
>
>> I think you meant the lack of a range check? Like mentioned here:
>>
>> + /* FIXME: We should signal an error below if START_BYTE
>> + etc. overflow the 32-bit unsigned data type. */
>>
>> I added buffer size check at parser creation time, and used casts to
>> uint32_t liberally, assuming the values never overflows and, so we
>> don’t need to handle the error at a million places. But I should have
>> added checks in ts_after_chang and other places where buffer size
>> could change. I’ll add checks in ts_after_change and other places, and
>> if the argumetns overflows uint32, it will set a flag (say,
>> buffer_too_large) in the parser object, and next time any lisp
>> function tries to use that parser, an buffer-too-large error will be
>> signaled. WDYT?
>
> Isn't there a way to make tree-sitter parse buffers larger than
> uint32_t? AFAIK, our buffers can be EMACS_INT large (especially with
> long log files), and tree-sitter should be fast enough to parse them
> incrementally?
I don’t think we want to get even near that size, since tree-sitter uses ~10x
the size of the buffer to store the parse tree. That’s probably why
tree-sitter’s author didn’t bother to use uint64.
Yuan
- Re: tree-sitter: Paths used for loading of language definitions, (continued)
- Re: tree-sitter: Paths used for loading of language definitions, Jostein Kjønigsen, 2022/10/17
- Re: tree-sitter: Paths used for loading of language definitions, Yuan Fu, 2022/10/17
- Re: tree-sitter: Paths used for loading of language definitions, Eli Zaretskii, 2022/10/17
- Re: tree-sitter: Paths used for loading of language definitions, Yuan Fu, 2022/10/17
- Re: tree-sitter: Paths used for loading of language definitions, Eli Zaretskii, 2022/10/17
- Re: tree-sitter: Paths used for loading of language definitions, Stephen Leake, 2022/10/17
- Re: tree-sitter: Paths used for loading of language definitions, Eli Zaretskii, 2022/10/17
- Re: tree-sitter: Paths used for loading of language definitions, Yuan Fu, 2022/10/17
Re: Some issues with the tree-sitter branch, Yuan Fu, 2022/10/17
Re: Some issues with the tree-sitter branch, Yuan Fu, 2022/10/17