Re: Locations suggest -- we're stupid

[Nikos Balkanas]

----- Original Message -----
From: Hans Aberg <address@hidden>
To: Help Flex <address@hidden>
Sent: Wednesday, January 09, 2002 1:01 PM
Subject: Re: Locations suggest -- we're stupid


> At 14:35 +0200 2002/01/08, Nikos Balkanas wrote:

[...snip...]
> >Yes I have done tests. Duplicating the yytext scan has an overhead. Used
> >yy_scanbuffer and scanned input from the buffer twice or once.
>
> Did you first, in the Flex source file main.c, check_options(), zip out
>   if ( do_yylineno )
>     /* This should really be "maintain_backup_tables = true" */
>     reject_really_used = true;
> recompile, and then do timings on a test with and without the %option
yylineno?
>
> If so, what did the test show, on which computer?

Nothing of the sort. Read from stdin into a buffer. yy_scanbuffer(). Fired
up yylex(). Timed it (not profiled it) running through the patterns once.
Then on EOF I yy_scanbuffer() again and throw yylex into a state that counts
up new lines. Timed that too. This is faster than scanning yytext for '\n's
using C. Showed consistent difference. By how much I don't remember. There
is no point to it. No one is arguing to remove yylineno.

[...snip...]

> >Processing time adds up. A slow lexxer can hog down the fastest parser.
> >One of the main advantages of flex is its speed. Let's keep it that way.
>
> The thing is that for the processing of the characters, the main
bottleneck
> of a stream is to get it off the hard disk. There are several buffers,
> possible involving character translation with conditionals before the
> character even enters the Flex generated lexer. But that probably does not
> affect the overall speed, as the CPU's are much faster than the hard disk
> can read the file.
>
> So therefore, it is good to have some profiling on modern computers done
> (which will behave differently than old ones).

I use overall time which is affected by I/O. One cannot do anything about
disk I/O and the system calls for doing it. One can hope only to make fast
software on top of that. Remember times are additive (Unless flex does I/0
on a seperate thread which currently doesn't). Then again you have people
that, realizing this,
work from memory instead of a hard disk (after the first pass use
yy_scanbuffer for subsequent passes). Furthermore, in development time when
one debugs a particular input, many OS's (linux for example) will cache that
input into memory (writing to disk, however, is the big hog, which cannot be
cached).

Sorry, no excuse for slow code :-)

> >> So one should not wasting time on trying to write efficient code, but
to
> >> write structured code, and optimize it wherever needed, if possible:
Much
> >> more precious than computer time is programmer time, which forces this
> >> development.
> >
> >Programming style. I like to write optimized code from the start, so that
I
> >avoid going back at it later on. Development is very subjective. One
person
> >can write code in 1/4 the time other people do.
>
> The thing is that computer are changing, so it is not worth doing
> optimizations, if the code is not executed often. Structured code is
easier
> to maintain, which is waht counts if programmer time is expensive.

Reminds me of Microsoft's approach: Bigger (and slower) is better. A faster
software will still be faster on any computer.
Flex is excellent at providing structure right off the box. Propably the
only language that can do without comments. I didn't know you can skip
structure when using flex :-). I see no tradeoff between structure and
efficiency with flex.

I consider that the surest way to improve as a programmer is to consider
one's
own time inexpensive. Then one can turn some really nice (reusable) code. Of
course that means putting extra time to it at off hours, initially.

However, this is of no interest to the list.

[...snip...]

> As for this discussion on error messages, I by that got aware of the
> information that it could absorb and make use of. So the error message
> information that looked as though being of general use, I fed back into
the
> Flex/Bison lists.
>
> But it was not exactly what is used in that IDE: Actually, it uses MacOS
> file descriptors, and some other stuff I did not mention.

So are you saying that the features proposed will not work with CodeWarrior?

> I have had conversations with other people, and the reason one is using an
> IDE and not GNU is the convenience. SO I want to feed the information back
> into GNU/Flex, so that people do not feel there is any advantage of that
> other stuff anymore.
>
>   Hans Aberg
>
>

In conclusion,

I wish you the best of luck. My argument based on the KISS principle is do
not implement the features
unless people really need it. Those features won't be used by me or any
of my friends. We are hooked on gcc and vim for now.

Furthermore in the best case (no bugs - completely isolated code) there will
be a delay in the parser generation (flex step is quite fast though). In
addition it will make for a larger flex source, more difficult to maintain.
Akim seems to think that it might be unfeasible. However, I am not the
maintainer. There are some things I wish I could see instead (egcs 2.91.66 -
wrong pattern generation, I/O on a seperate thread).

Nikos