Re: Syntax checks

[Lynn Winebarger]

Sorry for taking so long, I've been browsing guile.  I still haven't
thoroughly read Waddell/Dybvig's paper on modules yet, either,
but it's been 2 weeks so I thought I should go ahead and try to 
reply.  

On Sunday 14 April 2002 12:52, Marius Vollmer wrote:
> Lynn Winebarger <address@hidden> writes:
> >  The TODO list currently only lists 1.6 and "Eventually" as target
> > times.  I am interested in this particular task, but am still poking
> > around the source.  It's not entirely clear what the exact
> > difference between environments and modules is (or should be).
> 
> The environments are intended to provide the run-time data structures
> that implement the module system.
> 
      Actually, I had meant environments in the generic sense (what you
supply to code to provide values to variables) rather than any particular
implementation.  At the time I didn't realize there was an 'environment'
type in guile code.

> I myself am not entirely sure how to use the environments, and if we
> need their--although elegant--richness.  When developing the semantics
> of the module system, we should not try to move it into such a
> direction that it fits perfectly with the existing environments
> interfaces.  If it does fit, so much the better, but that should not
> be the main goal.

     Fine by me.  While in principle I think the idea of examinable 
environments is neat, we should think carefully about how features
would interact with a compiler.  I'd say, for non-debugging purposes,
variables/syntax should be visible only if it's declared "exportable",
and non-visible otherwise.
     I am in favor of Waddell/Dybvig's general approach, i.e. modules
provide another form of lexical scoping, evaluate to first class objects
and can be nested.  

> > Also, while I'm aware modules should act
> > orthogonally to objects in terms of introducing namespaces, it seems
> > to me the module system should allow constructs similar to the
> > imperative object-oriented languages.
> 
> Can you elaborate?  Are you talking about data hiding, inheritance,
> etc?
> 
     Yes.  How will generics interact with the module system? How
will classes interact with nested modules?
    I'll try to write some (currently non-interpretable) code to illustrate
what I'm thinking about and how it should evaluate.

> 
> >          The real question (lingering from at least late 2000) seems
> > to be whether lambda abstractions should delay expansion as well as
> > evaluation.  My first impulse is to say it shouldn't, that macros
> > are "evaluated" at read time.
> 
> Yep.  I think we should be careful about defining our 'times'.  'Read
>  [...]
> I think that following the 'read time' (of a Scheme program), we
> should always have a 'compile time' (even when we are going to execute
> the program right away).  During this compile time, Scheme macros are
> expanded.  Then might follow 'execute time'.
> 
      I took another look at R5RS for more insight.  Looks like 
syntax expansion, compilation, and evaluation should be separate stages
(logically if not implementationally).  This seems correct to me.

> > (define (foo x) (bar x))
> > (define (bar x) (+ x 5))
> > (define-syntax bar (syntax-rules () ((_ x) (+ x 5))))
> > 
> >     the 2 definitions of bar work "the same".  However, IMO, the second
> > definition should yield an  error in  (foo 4)  because it's evaluation time 
> > and bar evaluates to a macro,
> 
> Yes, and a block compiler might warn about this in advance.  (Both
> about redefining bar and about using bar as a function and later
> defining it as a macro.)
> 
> Hmm, I wouldn't say "bar evaluates to a macro".  This makes sense
> right now because of the way our evaluator works, but I think it wrong
> to say that 'bar' is evaluated, at all.  It is recognized by the
> compiler (or memoizer) as a macro.

           For a while I was thinking you were wrong, but upon more 
reflection I see you are right.  R5RS does specify them as separate
(Node: Variables; syntactic keywords; and region in r5rs.info).  Chez
5.0b the following interesting behaviour:
(define foo (lambda (x) (bar x)))
(define bar +)
(define-syntax bar (syntax-rules () ((_ x ...) (+ x ...))))
(foo 5) => 5
(bar 5)  => -5
bar or (cons bar '()) or (define bar +) 
      => error about use of keyword in non-operator position

     It's interesting because the old code continues to refer to the 
variable binding and yet you can't create a new reference to that location.
This seems to be the logical result of requiring the ability to specify syntax
in non-operator positions ("identifier-syntax").  
     My thinking is that while they're logically separate environments, they 
should
should be implemented in one table, where each entry has 2 possible entries
(one for syntax and one for variable).

> > and 5 is not "syntax".
> 
> I don't understand, could'ya explain?

    I meant it's a value, not a token.  The original message included
a bit about dereferencing a variable location and getting a macro, 
and that a (syntax-case macro anyway) should complain if it's not
acting on "syntax".     Actually, I should have said "4 is not syntax".
    Of course that's not really true as syntax-case is sloppy in what it
will accept to allow for recursive munging.

> >     Mainly, however, I see this as a kind of lexical scoping - if
> > you re-evaluated macros whenever they changed, you have a kind of
> > dynamic scope.  [...]
>
> Hmm, if I remember the discussion right, we were talking about two
> kinds of consistencies of a system, not about scoping disciplines.

     There's a difference?

> Consider the following simple model of interactive development of a
> 'system': you start with a set of files and load them into a freshly
> started Guile process.  You then make changes to the files and load
> them again.  Do this a couple of times.  At the end, you have a new
> set of files, and Guile process in a certain state.  You then start a
> second, fresh Guile process and load the new set of files.
> 
> When the two Guile processes must be in the same state, we would have
> "static consistency".  The state of the system is only determined by
> what is in the files that describes it.  When the two Guile processes
> are allowed to differ in their states, we would have "dynamic
> consistency".  The state of the system is determined by the operations
> performed on the Guile process (in our simple model, the only
> operation was loading a file).
> 
      To me, this description views the files themselves as the source 
of bindings (i.e. a scope and instantiation of a scope at the same time).
That's why the usage is backwards. Since files are dynamic objects (and
completely outside the interpreter's grasp), synchronizing with them is
a dynamic activity.  Consistency with what's been read (which is a static
concept as "what's been read" never changes except in being added
on to) is the static consistency.
    I would actually argue that this brand of "static vs. dynamic" prevents
modules from being completely orthogonal to directories and files.  I.e.
modules should be kept all in file (though more than one module per
file is fine).  Having one module in multiple files is asking for trouble.
Although it may be that the trouble doesn't need to concern the language
implementation and compiler optimizations.

> [...]
> For example, instead of magically reexpanding all uses of a redefined
> macro, we should simply require the user to reload or recompile all
> affected files, if he so desires.  The system can help by providing a
> list of affected files, and doing the whole reloading/recompiling upon
> a simple command.

    Sounds good to me.  

> Also, I think we should extend this to bindings in modules: when the
> list of exported bindings of a module changes (say), the current
> proposal is to automatically correct all existing uses of the affected
> bindings.  I now think it will be better to fix the list of imported
> bindings when executing a 'define-module' or 'use-modules' (if it
> survives) statement.  This is Dirk's signatures idea, and I now
> finally got it, I think.  When you want a changed export list to take
> effect, you need to reload/recompile the files that are affected by
> it.
> 
     I've looked at signatures.texi, but I don't see how this relates.
I'm not actually sure adding bindings to a module _after_ its definition/lexical
scope should be possible at all.  Redefinition of the entire module, yes. 
But then the old definition/bindings would be still in use by the other
modules that used it.
      Another issue - how would you direct the compiler to export C
"bindings" (e.g. a trampoline shared library and header file).  This
touches on something I've noticed while browsing - why are there 
so many smobs in the core interpreter?  Is it because you can't export 
macros to C automatically, or is there a deeper reason?

Lynn