|
From: | Gautam Kapoor |
Subject: | RE: performance bottleneck in yy_get_next_buffer function |
Date: | Tue, 15 Jan 2013 14:27:24 +0530 |
Hi Hans, This is true both for flex and flex++. Yes, my patch fixed that problem. Ideally, I would expect memcpy to be used instead of a for-do there. But Martin (attached email) suggests that it could be fixed by changing the buffer size. I have to try that. If one can choose an optimal buffer size (may not always be possible), then the effect of this for-do will be minimized. -regards Gautam -----Original Message----- From: Hans Åberg [mailto:address@hidden Sent: Thursday, January 10, 2013 5:00 AM To: Gautam Kapoor Cc: address@hidden Subject: Re: performance bottleneck in yy_get_next_buffer function On 9 Jan 2013, at 18:20, Gautam Kapoor <address@hidden> wrote: > I want to discuss a particular performance issue and how I tried to fix it. I > am wondering why it is not part of the default scanner generated by flex > because I think the developers must have seen this too. Flex is currently not developed or maintained, it seems. But a known problem is that rules that capture a lot of input slows the lexer down. So, for example, scanning for a comment spanning several lines is better done line by line. Does your patch fix that problem? Hans
--- Begin Message ---Subject: RE: performance bottleneck in yy_get_next_buffer function Date: Thu, 10 Jan 2013 14:36:35 +0530
Hi Gautam,
If your scanner spends a lot of time moving those buffers, then the buffer is too small for your maximum token length. Both can be defined somewhere, if I remember correctly.
The buffer should be much much bigger then the maximum token length.
regards,
Martin
> From: address@hidden
> To: address@hidden
> Date: Wed, 9 Jan 2013 22:50:15 +0530
> Subject: performance bottleneck in yy_get_next_buffer function
>
> Hi Compiler Experts,
> I am using flex to generate a scanner. It creates a yy_get_next_buffer() function in lex.yy.cc among other things.
>
> I want to discuss a particular performance issue and how I tried to fix it. I am wondering why it is not part of the default scanner generated by flex because I think the developers must have seen this too.
>
>
> When I run a particular test-case, I see a lot of time being taken by this function (yy_get_next_buffer). On running tools like "quantify" to measure performance, I found that these particular lines in yy_get_next_buffer() are taking a lot of time.
>
> /* First move last chars to start of buffer. */
> number_to_move = (int) ((yy_c_buf_p) - (yytext_ptr)) - 1;
>
> for ( i = 0; i < number_to_move; ++i )
> *(dest++) = *(source++);
>
>
> When I change this to the following, I get a huge improvement in performance.
>
> /* First move last chars to start of buffer. */
> number_to_move = (int) ((yy_c_buf_p) - (yytext_ptr)) - 1;
>
> memcpy(dest,source,number_to_move);
>
>
> -regards
> Gautam
>
> _______________________________________________
> help-flex mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/help-flex
--- End Message ---
[Prev in Thread] | Current Thread | [Next in Thread] |