Re: thoughts on peval

[Ludovic Courtès]

Hi Andy,

Thanks for taking the time to review the code, I greatly appreciate it!

Andy Wingo <address@hidden> skribis:

> (let ((x 0))
>   (call-with-values
>       (lambda () (if (zero? x) (values 1 2) (values 3 4)))
>     (lambda (a b)
>       (+ a b))))
> => 3

Ah, cool (which makes part of my previous message moot.)

> My goal is to have peval do well when compiling psyntax to produce
> psyntax-pp.scm.  However it does not currently work -- the peval
> aborts.  Looking into this issue, it seems that peval does not work for
> any of our .scm files (!!!).  The reason for this is that the code that
> looks to inline applications bails for a number of common procedure
> types, including module-ref forms.  Module-refs are mostly produced by
> macro expansion, such as the call to define-module* that starts most of
> our .scm files.

Oooh, ahem, good catch!

> I tried making peval leave module-ref calls alone, and it does let peval
> proceed, but then things explode: since there is practically no
> constraint on inlining, the code size and compilation time explodes.
> Any call to a procedure whose definition is known is eligible for
> inlining if any argument to the procedure is constant, including lambda
> arguments, regardless of the size of the procedure being inlined.

Well, top-level procedures aren’t inlined, right?

--8<---------------cut here---------------start------------->8---
scheme@(language tree-il optimize)> (define til (cut compile <> #:to 'tree-il))
scheme@(language tree-il optimize)> (peval (til '(begin (define (foo x) x) 
(define (bar x) (foo x)) (bar 2))))
$2 = #<tree-il (begin (define foo (lambda ((name . foo)) (lambda-case (((x) #f 
#f #f () (#{x 16303}#)) (lexical x #{x 16303}#))))) (define bar (lambda ((name 
. bar)) (lambda-case (((x) #f #f #f () (#{x 16301}#)) (apply (toplevel foo) 
(lexical x #{x 16301}#)))))) (apply (toplevel bar) (const 2)))>
--8<---------------cut here---------------end--------------->8---

So there must be something else going on?

> This concerns me a great deal.  We will need to work hard to
> retain/regain O(N) compile times.  Before peval, compile times (with a
> bootstrapped compiler) were roughly linear in program size.  If we
> diverge too much now, we won't be able to compile large programs.

Agreed.

> Also, I think that peval is probably inlining too much, at least by
> default.  By the time that code hits peval, all definitions in a file
> are in one big <sequence>.  Peval assumes those definitions are static,
> but that has not historically been the case.  At least by default, we
> should only inline statically-bound definitions.

This is not the case AFAICT–i.e., top-level definitions are *not*
eligible for inlining (this is a todo in optimize.scm and in my original
post.)

I was well aware of the risk of excessive inlining, but I thought that
this and the two other conditions to abort inlining would mitigate the
problem (the two other conditions being: any static args, and no
recursive calls in the residual code.)

> Finally I think that we should be more pragmatic regarding lexical
> set!.  One set! should not abort optimization of an entire form.

Sure.  My first approach was: let’s see if we can optimize common
functional idioms, and we’ll see later about imperative idioms, which
are much more difficult to handle.  Bailing out upon set! & co. was a
simple way to ensure peval wouldn’t do anything wrong with that code.

> I suppose the blockers for the next release would be the excessive
> inlining thing.  We should also fix toplevel inlining.  Ludo, what do
> you think?

I think you raise very good concerns, and I agree with the blockers.
However, there’s no top-level inlining AFAIK, so we should find out what
exactly goes wrong.

Thanks,
Ludo’.