Re: Serious performance issues with 1.9.0

[Andy Wingo]

On Mon 20 Jul 2009 14:48, Martin Ward <address@hidden> writes:

> On Sunday 19 Jul 2009 23:03, Andy Wingo wrote:
>> Discussed here:
>>
>>   http://article.gmane.org/gmane.lisp.guile.devel/8882
>>
>> I'm working on a fix, hopefully within a couple of days.
>
> Great! Let me know when it is ready, and I'll put it through
> its paces for you.

Well! Your code brought to light three limitations in the current code,
and two bugs.

The first limitation was that before, we allocated all set! variables in
a linear list within a function. Now they are allocated in boxes on the
stack. I increased the number of possible stack variables from 255 to
64K, but that turned out not to be necessary, since your set! variables
were often in ephemeral lexical scopes, so the local variable slots
could be reused in different parts of the function.

Then the first bug: sometimes the alpha-renamed variables from psyntax
were not globally unique. The algorithm was, "variable stem +
monotonically increasing integer". Obviously that doesn't necessarily
work if the variable stem ended in an integer. I was aware of this, but
somehow it hadn't bitten anyone until compiling your code. Now it is
fixed.

At this point I could compile ALL.scm.

Then the second limitation: the default VM stack is too small, and we
have no overflow handler to increase it. So I raised the default VM
stack size. We'll lower it back down again, perhaps, once we implement
overflow handlers.

Then the second bug: ALL.scm compiled, and the first could
prog_to_spec_TEST tests passed, but then I got an "invalid instruction"
error. But the function that produced the error disassembled fine...
then I noticed the function was around 70K bytes in length, close to
another limit: that jumps are encoded in a 16-bit value. It didn't take
too much more poking to find out that a forward jump of 64823 bytes was
being interpreted as a backwards jump, because it's a signed 16-bit
value. I added some more checks in the assembler to catch this at
compile-time.

Which brings us to now. ALL.scm doesn't compile any more because of that
16-bit limit, which is now caught correctly. I am afraid we will have to
move to 32-bit words as Ludovic has suggested all along, because the
proliferation of short and long addressing modes is getting too much. I
guess that's the next thing on my plate, then...

Regards,

Andy
-- 
http://wingolog.org/