Re: Which lexer do people use?

[Christian Schoenebeck]

On Montag, 6. Juli 2020 14:00:17 CEST Maury Markowitz wrote:
> > On Jul 4, 2020, at 6:46 AM, Christian Schoenebeck
> > <schoenebeck@crudebyte.com> wrote: The only thing that people are missing
> > once in a while on scanner side is unicode support
> 
> I am missing the capability to support grammars with optional spaces between
> elements. In BASIC:
> 
>    FORI=1TO5
> 
> is equivalent to:
> 
>    FOR I=1 TO 5
> 
> I have yet to see an elegant way to model this in Flex. This topic comes up
> time to time here, so I don't think Unicode is the "only" thing.

You would simply add a RegEx pattern & rule like this:

[ \t\r\n]*  /* eat up white space */

and/or the same for 'any other unknown input':

.  /* drop anything else i.e. 'unknown' input */

Best regards,
Christian Schoenebeck