[Axiom-developer] Axiom Design Musings

[daly]

A primary goal of the LP rewrite is to make it much easier to
maintain and modify Axiom.

In particular, the current LP focus is on the interpreter code.

HISTORY

Historically the interpreter was written by several people in a
mixture of lisp styles. Some of the early work was done in Maclisp.
Another layer, called Meta, was added to study language design.  A
domain-specific language called Boot was created in Meta, which had a
syntax similar in spirit to the Spad language. When the system was
ported to VMLisp a layer of macros and functions covered some
differences. As each new lisp became available an additional layer of
macros and functions were created.

Meanwhile the world had created a well-defined standard language,
Common Lisp. More macro work was done and eventually Axiom was easily
ported to about a dozen Common Lisp implementations. IBM contracted
with William Schelter to create a Common Lisp called Austin-Kyoto
Common Lisp (AKCL), now called Gnu Common Lisp (GCL) that was 
specifically optimized to run Axiom.

THE PRESENT SITUATION

The current effort aims at "strength reducing" the code. That is, to
remove all of the artifacts of the historical layering, rewriting it
into Common Lisp.

The Meta language is gone. The Boot language is gone.  Removing Meta
and Boot removed many files and reduced the Makefile complexity.  It
also improved build times considerably. In the long term the build
system will be a Common Lisp function.

What remains is lisp code with many layers of macros and functions
that still mimic older lisp versions. Work is being done to remove the
Maclisp and VMLisp idioms. These are not widely known and the manuals
for these languages are considered collectors items.

Portions of the interpreter are still in machine-generated form
waiting to be rewritten by human hand. This effort will likely consume
most of 2015.

Overall the code is converging on simplified, by-the-book Common Lisp.
Soon the only reference document needed for Axiom internals will be
the Common Lisp book. But this is just a plateau for future
improvement.

THE NEAR FUTURE

Axiom internals are still quite opaque.

Consider the global variable situation. Shared, mutable state is
now known to be a source of bugs and a barrier to understanding.

Axiom has a whole sea of global variables, the so-called dollar
variables, which hold shared state information. Mutable, shared state
makes it very difficult to maintain and modify code. It would be good
to document the state but it would be better to restructure the code
to minimize it.

Some of the dollar variables represent truly global state such as
which stream is being used for output. Other dollar variables are
"locally-global" state shared within functions such as the undo
code. The first step will be to distinguish truly global state
from locally-global state. 

A near-term approach would be to encapulate locally-global state
in defstructs and pass the structure in as an environment argument
to the function. That would remove a lot of dollar variables and
make it clear that the state is an argument to the function.

THE NEXT HORIZON

The dollar variables will be restructured "in the small" but there
is a design question about system structure "in the large". For
instance, there is an "undo" facility. One design goal would be
to make it possible to maintain and modify this functionality
without impacting the rest of the system. How can this be done?

Object-oriented programming is one possible approach. Objects can
encapsulate state making it clear which functions can and cannot
manipulate it. Lisp has at least two approaches to creating
object-like behavior, either using Let-Over-Lambdas (LOL) or the
Common Lisp Object System (CLOS). 

Functional programming is also possible. Data structures can be
reimplemented as immutable objects and procedures can be rewritten
as pure functions. A side-effect would be to make "undo" trivial.

We could also consider "lifting" the code into Spad, trying to
make Axiom mostly self-hosting. This may be a bridge-too-far though
as Spad has abstract algebra, a good mathematical scaffold, but a
poor notion of low-level implementation concepts. Additionally,
making Spad self-hosting brings back the need-Axiom-to-make-Axiom
circle that was fixed in 2001.

It is not clear what the most understandable alternative might be.
Whatever is it, the focus is on clarity. The design needs to be
easy to explain, maintain, and modify.

Tim