Re: common per-token info

[Akim Demaille]

>>>>> "John" == John W Millaway <address@hidden> writes:

John> Akim: Just for the sake of example, can you give us a scenario
John> for which you need to track the "width" of each token, (where
John> width != yyleng) ?

Strings come to my mind.  You have to accumulate.  And what do you
count: chars or bytes?

But I might not have understood the question: if you refer to the
width I was referring to before, I meant that there are (at least) two
schools: one is

        location = start-pos + end-pos

the other being

        location = start-pos + width