Re: Nested comments

[Hans Aberg]

At 09:48 +0200 2003/07/03, Akim Demaille wrote:
> > I implemented nested [* ... *] comments by the code below, but it turns out
> > to be slow with many nestings, and I do not know why. Is there a way to
> > make it faster?
...
>I personally do the following:
>
>----------------------------------------
>
>"/*"        comment_level++; BEGIN STATE_COMMENT;
>
><STATE_COMMENT>{ /* Comments. */
>  "*/" { /* End of the comment. */
>    if (--comment_level == 0)
>      {
>       yylloc->step ();
>       BEGIN INITIAL;
>      }
>  }
>
>  "/*"         comment_level++;
>  [^*/\n\r]+   ;
>  {eol}+       yylloc->lines (yyleng);
>  .             /* Stray `*' and `/'. */;
>
>  <<EOF>> {
>    std::cerr
>      << *yylloc << ": unexpected end of file in a comment" << std::endl;
>    exit_set (exit_scan);
>    BEGIN INITIAL;
>  }
>}
>
>----------------------------------------
>
>The difference I can see, besides your better efforts at matching long
>non-\n tokens, is that I don't match \n in the main RE, since that
>could make huge tokens, which is certainly bad for performances,
>especially if you do have long comments (Flex has to allocate, cache
>penalty blah blah blah).

Thank you, Akim:

This does the trick: Gulping up 600 lines of nested comments now takes an
insignificant amount of time relative 200 lines of proof proving.

Here is a version that uses [* ... *] comments instead of /* ... */:

"[*"  { BEGIN(comment); comment_level = 1; MLIparserlineno0 = yylineno; }
<comment>{ /* Comments. */
  "*]" { /* End of the comment. */
    if (--comment_level == 0) {
            BEGIN INITIAL;
    }
  }

  "[*"        { comment_level++; }
  [^*[\n]+    {}
  \n+               {}
  .           { /* Stray `*' and `['. */ }

  <<EOF>> {
    std::cerr << "Error: Nested comments not properly closed at end of
file.\n";
    BEGIN(INITIAL); return YYERRCODE;
    /* exit_set (exit_scan); */
  }
}
"*]"  { std::cerr << "Error: Too many comment closings *].\n";
        BEGIN(INITIAL); return YYERRCODE; }

  Hans Aberg