[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090)
From: |
Keith OHara |
Subject: |
Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090) |
Date: |
Sun, 01 Jan 2012 02:05:49 -0800 |
User-agent: |
Opera Mail/11.60 (Win32) |
On Sun, 01 Jan 2012 01:40:36 -0800, <address@hidden> wrote:
Our lexer has been written with the decision of using non-compressed
tables and without backing up.
Off topic, but maybe interesting to you: I don't think that decision was ever
implemented. I don't see any "%option full" or similar that would generate
non-compressed tables. For at least a few years lexer.ll did generate backup states; I
avoided them as part of the purge of warnings. Also, lexing takes a tiny fraction of
LilyPond's execution time. If you ever want to use a method that requires backing up, I
think you should.
I spent more than a day's worth on doing
utf-8 right in the grammar. That's pretty pointless. It also means
that we need to provide an error path for every item containing
non-UTF-8 characters in order to get a UTF-8 related error message
instead of something more mysterious.
Okay, then.
Consider a comment in your case/switch statement that points to some reference
on the various types of UTF-8 validators.
- Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090), dak, 2012/01/01
- Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090),
Keith OHara <=
- Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090), dak, 2012/01/01
- Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090), dak, 2012/01/01
- Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090), dak, 2012/01/01
- Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090), k-ohara5a5a, 2012/01/01
- Re: lexer.ll: Warn about non-UTF-8 characters (issue 5505090), dak, 2012/01/02