Re: Flex recognition of "split" keywords

[John R Levine]

> I looked a bit at your examples, and it makes me wonder why somebody would
> invent such a language: Especially the feature where spaces are prescanned
> out. What would the advantage of it be? It surely does not help human
> parsing.

Remember that Fortran was designed in the 1950s, before any modern
compiler theory was invented, for an environment with 80 column punch
cards and 200 bpi magtapes.  The ability to squash as much as possible
onto a single card was probably considered a feature.

> >     a = 10 e2               // 10e6 is a number
> >
> >     do 10 e2 = 1,10 e3      // 10 is a statement number, e6 is
> >                             // a name, 10e3 is a number
> (Your comments are using e6 instead of e2. :-) )

Well, yeah, I can't type.

> A similar method might be used in the example above, i.e., the parser would
> get the tokens "10" and "e2" and decide whether they should be a number or
> two separate tokens.

It's a lot messier than that -- floating point constants can use "d" or
"e" and can have a signed exponent, e.g 12e+34 or 12.34e-56.  They also
look similar enough to fields in format statements (10e8.2 says ten fields
of eight characters each with a two-digit decimal) to cause confusion.
The only reasonable way to lex Fortran is with advice to the lexer so it
knows when it's looking for a statement number, when it's looking for a
general number, and when it's parsing a format statement.

With a prepass to handle the fixed length statement number and line
continutation fields, to decide whether a statement is an assignment or
something else, to recognize quoted strings and remove all the other
blanks, then , with lots of feedback from the parser to set %x modes, you
could probably do the last bit of tokenization in flex.  But why bother?

> This way the "contexts" are sorted out by the parser, not the lexer.

That might be possible, but the result would be impossibly ugly and hard
to debug or maintain.

Regards,
John Levine, address@hidden, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Sewer Commissioner
"I dropped the toothpaste", said Tom, crestfallenly.