Re: Flex recognition of "split" keywords

[Hans Aberg]

>> I looked a bit at your examples, and it makes me wonder why somebody would
>> invent such a language: Especially the feature where spaces are prescanned
>> out. What would the advantage of it be? It surely does not help human
>> parsing.
>
>Remember that Fortran was designed in the 1950s, before any modern
>compiler theory was invented, for an environment with 80 column punch
>cards and 200 bpi magtapes.

I know, I used them in the 70'ies.

>  The ability to squash as much as possible
>onto a single card was probably considered a feature.

You are right, that is probably the reason: Perhaps people, as nowadays in
telephone messages, used special ways to squeeze the information. Then,
once that language has been created and become popular, the desire for
backwards compatibility gets one stuck with the features turned quirks.

>> A similar method might be used in the example above, i.e., the parser would
>> get the tokens "10" and "e2" and decide whether they should be a number or
>> two separate tokens.
>
>It's a lot messier than that -- floating point constants can use "d" or
>"e" and can have a signed exponent, e.g 12e+34 or 12.34e-56.  They also
>look similar enough to fields in format statements (10e8.2 says ten fields
>of eight characters each with a two-digit decimal) to cause confusion.
>The only reasonable way to lex Fortran is with advice to the lexer so it
>knows when it's looking for a statement number, when it's looking for a
>general number, and when it's parsing a format statement.
>
>With a prepass to handle the fixed length statement number and line
>continutation fields, to decide whether a statement is an assignment or
>something else, to recognize quoted strings and remove all the other
>blanks, then , with lots of feedback from the parser to set %x modes, you
>could probably do the last bit of tokenization in flex.

Another idea that comes to my mind is to make a prepass that inserts the
spaces.

>  But why bother?

Right. I am only moderately interested in Fortran quirks.

>> This way the "contexts" are sorted out by the parser, not the lexer.
>
>That might be possible, but the result would be impossibly ugly and hard
>to debug or maintain.

One would have to split up tokens into "atoms" which can be picked together
by the parser.

The point would be that Fortran is an old language that does not change. So
if one is succeeding with it once, there would be no need for "maintaining".

BTW. Is there not a GNU Fortran compiler available. -- Then one can use its
parser as a model.

  Hans Aberg