Re: precise input location

[Nicolas Sceaux]

Han-Wen Nienhuys <address@hidden> writes:
>
> Right now, parser.yy sets the input location of music expressions
> directly (using set_spot()). This method is flawed, since the parser
> occasionally has to look ahead for parsing correctly, thus putting the
> origin property in the wrong spot. If you want to attack this problem,
> you would have to modify the lexer to store the exact input locations
> of each token (including the end); there's no need to store the input
> string itself, since that is defined by the begin and end pointer of
> the token. (Locations are stored as pointers into a memory copy of the
> input file.)
>
> Then it should not be hard to make the parser use the location of
> every token. A music expression is located at the start/end of its
> tokens.

Would that mean to change the token types from 

  %union {
        Book *book;
        Output_def *outputdef;
        SCM scm;
        String *string;
        Music *music;
        Score *score;
        int i;
  }
  
to something like:
  
  %union {
        Book *book;
        Output_def *outputdef;
        SCM scm;
        String *string;
        Music *music;
        Score *score;
        int i;

        SCM_token *scm_token;
        String_token *string_token;
        int_token *i_token;
  }

with for instance;

  struct String_token {
    String *val;
    Input origin;
    Input end;
  } /* or a set of classes */

Then in the lexer:

  *yylval.string_token->val = YYText ();
  *yylval.string_token->origin = ???;
  *yylval.string_token->end = ???;

?

In order to propagate origins to non-terminals, other (non-terminal)
types should also have the origin and end fields.

Am I missing something more straightforward?

nicolas