Re: [Axiom-developer] Of possible interest to Axiom

[Stephen Wilson]

address@hidden writes:

> Stephen,
>
> (sorry for the delay but I have a major power outage due to Ike,
> which will likely last until friday so online time is limited).
>
> Actually I think this is an excellent idea. If there were
> defcategory, defdomain, and defpackage macros that could be used
> as compile targets it would greatly simplify the spad compiler.

For any infix syntax, all that is needed is a parser which emits lists of tokens
instead of lists of symbols (so that the tokens can contain line/column info).

The current implementation takes care of type checking and code generation.

> I'd like to see a one-to-one mapping of spad language constructs
> to CLOS constructs. It would make it possible to expand the compiler
> to handle things like dynamic extends easily. These would be done
> with new defgeneric definitions are either compile or runtime. The
> underlying lisp system could simply make the magic happen since CLOS
> generics can be written at any time.

The current code uses CLOS as the basis for its ast.  There are classes to
represent domains and categories.  So a particular domain like Integer would be
an instance of the CLOS class DOMAIN, and a domain constructor like List would
be an instance of the class FUNCTOR.  Expressions like List(Integer) are of type
FUNCTOR-APPLICATION, etc. 

One of the main issues in designing this system is developing a protocol for
building, inspecting, and modifying domain instances.  The typechecker uses this
protocol to build the AST from s-expressions, the code generator uses it to
resolve abstract function calls and lay out efficient call tables, etc.  A
program to construct a graph of the algebra would use one of the iterators
defined over an objects set of supersignatures, etc.

The overall result is a new language extension to lisp, written in lisp, that
supports Spad-like semantics -- just like CLOS itself.  Even though we could
have a distinct CLOS class to represent every domain and category in the system,
I have not yet found a case where there there is a clear advantage.  I thought
about that route and decided that I wanted all of the information and behaviour
for a domain to be contained in its instance, rather than define properties
`from the outside' using specializations on generics.

What we can do though is define entirely new types of domains.  For example,
there are a few primitive domains that I use to test the system.  They get
defined by a DEFPRIMITIVE macro and result in objects of type PRIMITIVE-DOMAIN,
or PRIMITIVE-FUNCTOR (which themselves are sub-classes of DOMAIN and FUNCTOR).
The signatures and declarations are typechecked just like normal user domains.
The main differences are that their function bodies are plain lisp, and you can
specify a few extra options that are not available to normal domains (like what
the lisp type of the carrier is, or provide a macro expander to inline primitive
expressions during code generation, etc).  Here, the API defines generics we can
specialize on to accommodate these difference.

One place where there is a direct mapping is wrt expressions.  For example,
there is a CLOS class which represents IF statements.  The typechecker for
expression statements is just a bunch of methods on the generic CHECKEXPR.

It is absolutely trivial to add new expressions.  But I need to spend a lot more
time thinking about what the primitives should be.  For example, should I follow
Aldor and include GOTO in the language as a primitive for iteration? 


> If the CLOS mappings are defined it becomes possible to unify the
> interpreter and compiler. One could even consider playing with the
> MOPs to handle add-chain lookups. 

I spent a fair bit of time a while ago trying to use the MOP to implement
spad-like semantics, but I was not happy with the result.  I have not
implemented sub-domains yet, but there should not be any complications (just
questions about semantics.

> It would even greatly expand the error message and debugging ability
> since it would be trivial to add wrappers and whoppers.

An interpreter and debugger which operates over the expression AST should be
relatively straight forward.  Given a parser which generates lists of tokens
with line/number information we could have a very interactive and productive
environment.

We can supply our own implementation of wrappers and whoppers which take into
account the calling conventions of domain functions.


> So the upsides are cleaner semantic definitions, simpler lisp code,
> a clearly defined dynamic extend mechanism, compiler/interpreter
> unification, better error and debugging. I'm hard pressed to think
> of a downside.

I hope the explanations above clarify the approach I have taken.  I think it
will provide all of the advantages you list.

> I really like this idea a lot. I want it now. When can I have it? :-)

I hope soon.  There are a few things I really need to do before I can release it
for experimentation.

I would like to implement `inner domains' so that we can have domain valued
exports.  The mechanics are in place already, I just need to write the code.

Having that, I can write a package similar to Aldors "Machine" package, giving a
full set of primitive types to work with.

Currently, the only way to do iteration is via recursion.  I think I might
introduce GOTO, and implement generators or simple coroutines down the road.

The last, and most important issue, is to firm up a user package which gives a
public environment to define and evaluate expressions without having to be in
one of the internal packages of the compiler.  I have simple browser functions
defined so you can type "(browse 'monoid)" and get output describing the
category.  There is also a macro to call domain functions from lisp code without
needing to know the mangled symbol name that actually implements the function:
"(domcall integer + (-> (% %) %) 2 3)".


So a lot of the basics are there.  Just a few more cycles and I should have
something that I feel comfortable with releasing.


Thanks for the feedback on this!  Its very much appreciated!

Steve