[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: local-eval on syntax-local-binding, bound-identifiers

From: Andy Wingo
Subject: Re: local-eval on syntax-local-binding, bound-identifiers
Date: Mon, 16 Jan 2012 12:01:53 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux)

Hi Mark,

Thanks for the review!  My summary comments are at the bottom.

On Mon 16 Jan 2012 04:30, Mark H Weaver <address@hidden> writes:

> [... skipped the first patch, which looks very well implemented, though
>      I'm still not sure that we should be exposing this in our API ...]

Yeah, not sure what to do with it.  There is also the question of what
to do with syntax parameters; something that's easier to ask (and
answer) on master, though.

>> +                     (cons (wrap (car symnames)
>> +                                 (anti-mark (make-wrap (car marks) subst))
> ***** Why are you adding anti-marks here?

As the changelog noted (and a comment should have noted ;), the
identifiers are anti-marked so that syntax transformers can introduce
them, as-is.

The purpose of this procedure is to get a list of identifiers, and to
capture some subset of them.  It will do so by introducing references to
them in the expansion of some macro.  However they are not introduced
identifiers: they come from the code itself.  They are input the macro,
and as such need an anti-mark.

The anti-mark will be stripped from the expansion when the transformer
that called `bound-identifiers' returns.

As the idea of marks and anti-marks is part of the definition of
syntax-case, on a semantic rather than an implementation level, this
aspect of `bound-identifiers' probably needs to be documented.

>> From ddea51310227155e3771c3e6acbbecf24dc74c42 Mon Sep 17 00:00:00 2001
>> From: Mark H Weaver <address@hidden>
>> Date: Tue, 3 Jan 2012 04:02:08 -0500
>> Subject: [PATCH 3/3] Implement `local-eval', `local-compile', and
>>  `the-environment'
> ***** In general, when you make substantial changes to one of my
> patches, please remove my name as the principal author.  Instead, please
> say something like "Based on a patch by Mark H Weaver <address@hidden>"
> in the log entry.

The majority of the lines are yours; though it is difficult to assign
ultimate authorship here, to my mind it was quite fair to put you as the
primary author.  But, as you like.

>> +Also, @code{(the-environment)} does not capture +lexical bindings
>> that are shadowed by inner bindings with the same name, +nor hidden
>> lexical bindings produced by macro expansion, even though +such
>> bindings might be accessible using syntax objects.
> ***** I guess this last sentence should now be removed.  Also, remember
> that if pattern variables aren't reimplemented, this limitation needs to
> be documented.


>> +(define (partition-identifiers ids)
>> +  (let lp ((ids ids) (lexical '()) (other '()))
>> +    (if (null? ids)
>> +        (values lexical other)
>> +        (call-with-values (lambda () (syntax-local-binding (car ids)))
>> +          (lambda (type val)
>> +            (cond
>> +             ((eq? type 'lexical)
>> +               (lp (cdr ids)
>> +                   (acons (car ids) (datum->syntax #'here (gensym))
>> +                          lexical)
>> +                   other))
>> +             ((and (eq? type 'local-macro)
>> +                   (procedure-property val 'identifier-syntax-box))
>> +              => (lambda (id)
>> +                   (lp (cdr ids)
>> +                       (acons (car ids) id lexical)
> ***** You are putting inconsistent things in the cdr of each alist
> entry.  In the `lexical' case above, you make a gensym, and here you're
> putting the id of the box variable.  However, in the code below, you are
> not using this box, but rather just using these identifiers for
> temporaries.

Ah, indeed.  I did say it was very preliminary, no?  :-)

>> +         (lambda (lexical other)
>> +           (with-syntax ((module (datum->syntax #'here (module-name 
>> (current-module))))
> ***** This is wrong.  You want the module from the `scope'
> syntax-object, not the (current-module).

Yes, you are right.  However, here I think we need another primitive...

> ***** Here you are always making new boxes, even for previously-captured
> variables that have already been boxed.  Therefore boxes become nested
> and cost O(n) to access, where N is the nesting depth.

Good catch, that was certainly not the intention.

> More importantly: I notice that you are not stripping the psyntax wrap
> from identifiers placed within the wrapper procedure above.  There are
> certainly benefits to that, but remember that the wrapper procedure will
> in general be serialized to disk and evaluated in a different Guile
> session, where the gensym counters have been reset.

Of course, like all macros!  The forgeable gensym issue is something we
have in Guile, more generally, that needs a broader solution.

> Secondly, you are now serializing a psyntax internal data structure to
> disk (the wraps), and thus this now effectively becomes part of the ABI.

This is already the case.

>> +        ((module? e)
>> +         ;; Here we evaluate the expression within `lambda', and then
>> +         ;; call the resulting procedure outside of the dynamic extent
>> +         ;; of `eval'.  We do this because `eval' sets (current-module)
>> +         ;; within its dynamic extent, and we don't want that.  Also,
>> +         ;; doing it this way makes this a proper tail call.
>> +         ((eval #`(lambda () #,x) e)))
> ***** This was my mistake, but since I'm already marking up the code:
> the `lambda' wrap above needs a `#f' before `e' to force expression
> context.

OK.  (Note though that (eval X e) does indeed evaluate X in tail

> For the record, I still think it's better for `the-environment' to be
> implemented within psyntax as a core form.  It's a fundamental syntactic
> construct with clean semantics, and it belongs in psyntax with its
> brethren.  Your desire to remove it from psyntax has caused you to add
> far less elegant interfaces that have been hastily designed, and that
> may not even be sufficient for a full implementation of
> `the-environment' that captures mutually-recursive local macros.

In pursuit of the goal of agreeing on a strategy, I would like to
convince you that you are wrong on all of these points :)  So, in that
spirit, I argue:

`the-environment' is not fundamental: it can be implemented in terms of
simpler primitives.

`the-environment' does not have clean semantics, inasmuch as it has
nothing worthy of the name, not yet anyway.  The lambda calculus, the
scheme language, even the syntax-case system have well-studied semantics
(denotational and/or operational), and lots of experience.  The same
cannot be said of `the-environment'.  It is great that we are pushing
the boundaries of Scheme here, but I think it's hubris to think that
we'll get it right the first time.

Implementing the-environment in psyntax also poses implementation
layering problems: it uses wrap hacks to expand out to private forms
from ice-9 local-eval.  Instead, this implementation goes with the grain
of the expander, allowing the-environment to be implemented as a macro,
with the normal scoping rules for syntax objects.

It also seems to me that stripping identifiers down to their symbolic
values is the wrong thing to do.  It has a bad smell.

I readily grant that the interfaces I posted were "hastily designed",
though there is some prior art in Racket (nb, a different implementation
that's not psyntax).  However, if they do prove to have maintainable
semantics, I really do prefer to maintain that (in the syntax expander)
rather than `the-environment'.  Feedback here is much appreciated on the
form of these interfaces.

I think it's possible that this approach could work for mutually
recursive macros.  The key to note here is that the essence of the
mutual recursion is in the expansion of the syntax transformers (the
rhs), and that was already done within a proper recursive environment.
Later when the expansion produces, or would produce:

  (let*-syntax ((x proc-recursive-with-y)
                (y proc-recursive-with-x))
    (x ..))

the transformer procedures are already expanded (and evaluated).  So the
thing is that the bindings from x and y to their transformers must be
visible to the expression being local-eval'd, not to the transformer
procedures themselves.

Have I convinced you now? :-)



reply via email to

[Prev in Thread] Current Thread [Next in Thread]