Re: Nested comments

[Hans Aberg]

At 18:15 -0400 2003/07/05, address@hidden wrote:
>>It would probably be nice with a line counting
>> method that does not have to rescan the match.
>
>As it is now, line counts are nearly free if you
>separate the rules that could match newlines from the
>rules which could never match newlines. The mechanism is
>best suited to scanners which discard whitespace.

I realized that too. I have rule:
  [[:space:]]+      { /* Skip white-space. */ }
It should probably be split up into two rules:
  [ \f\r\t\v]+  {}
  \n+  {}
or something.

Anyway, I think the wisdom of this and some form of the Akim example should
go into the manual, if not already there. (I think the manual only mentions
non-nested comments.)

>>BTW, what happens if the match does not fit into the
>>buffer, which might happen if the comment is very long?
>
>The buffer grows with realloc(). However, recall that
>the default buffer size is 16k.

It is clear that the scanner chokes long before that 16k limit: The amount
of nest comments I used to test my example code was about 16k, but because
of the nesting, most rule matches should actually be a great deal smaller.
It would be good to know exactly what is the bottleneck. -- Perhaps you
have something that grows in say quadratic time complexity and needs to be
changed.


  Hans Aberg