Re: Some issues with the tree-sitter branch

[Yuan Fu]

> On Oct 16, 2022, at 10:37 PM, Po Lu <luangruo@yahoo.com> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>> I think you meant the lack of a range check? Like mentioned here:
>> 
>> +  /* FIXME: We should signal an error below if START_BYTE
>> +     etc. overflow the 32-bit unsigned data type.  */
>> 
>> I added buffer size check at parser creation time, and used casts to
>> uint32_t liberally, assuming the values never overflows and, so we
>> don’t need to handle the error at a million places. But I should have
>> added checks in ts_after_chang and other places where buffer size
>> could change. I’ll add checks in ts_after_change and other places, and
>> if the argumetns overflows uint32, it will set a flag (say,
>> buffer_too_large) in the parser object, and next time any lisp
>> function tries to use that parser, an buffer-too-large error will be
>> signaled. WDYT?
> 
> Isn't there a way to make tree-sitter parse buffers larger than
> uint32_t?  AFAIK, our buffers can be EMACS_INT large (especially with
> long log files), and tree-sitter should be fast enough to parse them
> incrementally?

I don’t think we want to get even near that size, since tree-sitter uses ~10x 
the size of the buffer to store the parse tree. That’s probably why 
tree-sitter’s author didn’t bother to use uint64.

Yuan