[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: How to decide what to put in the lexer and the grammar respectively?
From: |
Hans Åberg |
Subject: |
Re: How to decide what to put in the lexer and the grammar respectively? |
Date: |
Mon, 18 Feb 2019 00:10:13 +0100 |
> On 17 Feb 2019, at 23:10, Peng Yu <address@hidden> wrote:
>
> This lexical tie-in creates feedback from the parser to the lexer. So
> the lexer cannot be tested standalone.
>
> But the principle of separating lexer and parser is to make parser
> builtin on top of the parser. Is there something that can avoid the
> feedback yet still allow context-dependent parsing? Alternatively, how
> to just testing the lexer without having to get the parser involved?
The LARL(1) that Bison uses is for context free grammars only, so contexts must
involve switches somehow. Think of a definition which changes a name into an
identifier, implemented by putting it on a (typically stacked) lookup table,
which the lexer checks, and returns its token value to the parser. A grammar
definition alone would require an attribute grammar.
Re: How to decide what to put in the lexer and the grammar respectively?, Simon Richter, 2019/02/17