RE: [Axiom-developer] Terms of Surrender (was: BAD tim)

[Bill Page]

[Ooops, got too excited and hit the wrong key and sent this
prematurely ... please excuse the previous partial email.]

Tim,

I feel rallied by William Sit's support, so here comes another
defensive action.

On November 6, 2005 1:55 AM you wrote:

> 
> > First about Boot. My (admittedly poor) understanding of 
> > Boot is that it is an intermediate language between Lisp
> > and Spad (and there are other intermediate languages Shoe,
> > Meta, etc as well). I agree that to build a truly high
> > level language like Spad/Axiom/Aldor, intermediate
> > languages are necessities and not evils. It's much like
> > what a mathematician does all the time (and most original
> > developers are mathematicians, in addition to being 
> > computer scientists): if the
> .....[snip].....
> 
> sorry, but you are reasoning by wrong analogy.
>

William's analogy seems perfectly reasonable to me. That
is exactly the kind of "mindset" and approach that seems
consistent with Axiom's origins. The point is building a
set of concepts, often embedded in a new language (Yes,
mathematicians do invent new language all the time!) which
make expressing the ideas and proofs that you really want
to express, that mush easier. Of course there is an overhead
for learning the new language, but in mathematics that is
often part of the "fun" and when it's done you realize that
you have made some important intellectual gain in power.

The situation of Boot and Lisp is similar because the
abstractions and concepts implicit in Spad would be quite
obscure if one attempted to express them directly in Lisp.
Boot *does* provide an intermediate ground between the
concrete implementation in Lisp and the abstraction of
Spad - just as Dick Jenks apparently intended.

> boot is not an intermediate step between lisp and Spad.
> boot is a syntactic sugar coating on lisp that eliminates
> parentheses in favour of "piling" (indenting) code.

Tim, I am amazed that you would make this claim. It is quite
demonstratably incorrect. The use of parenthesis or not in Boot
is *completely* besides the point. Adding braces (as was done
in Aldor) is a trivial syntactic difference and *that* is what
I might agree could be characterized as simple sugar coating.
But the difference between boot and lisp, and the relationship
between boot and Spad is obvious. Even Scot Morrison seems to
hold this view. Just consider the example cited by Clifford Yap
many emails ago:

Boot language:

intloop () ==
    mode := $intRestart
    while mode = $intRestart repeat
      resetStackLimits()
      mode := CATCH($intTopLevel,
            SpadInterpretStream(1, ["TIM", "DALY", "?"], true))

Lisp:

(defun |intloop| ()
 (prog (mode)
  (declare (special |$intTopLevel| |$intRestart|))
   (return
    (progn
     (setq mode |$intRestart|)
     ((lambda ()
       (loop
        (cond
         ((not (equal mode |$intRestart|))
           (return nil))
         (t
          (progn
            (|resetStackLimits|)
            (setq mode
              (catch |$intTopLevel|
               (|SpadInterpretStream| 1
                 (list 'tim 'daly '?) t)))))))))))))

--------

Who could claim that the Boot code has a direct (sugar coating)
relationship to Lisp? Who would miss the similarity between
Boot and Spad?

> it adds no new functionality

Since when does any high level language "add functionality"?
You should not expect it too. Fortran does not add functionality
to Assembler language. If anything it specifically removes
functionality, trading-off extreme flexibility for a more
congenial model of computation. No one could argue that the
development of high level programming languages was some kind
of mistake.

> and does not help you reason about programming any more than,
> say, python or perl.

Here I also completely disagree with you. I *do* think that
Boot, Python and Perl do help you to reason about programming.
I don't mean in the "nuts and bolts" kind of way of say, a

[message continues here]

'while' statement version a conditional jump instruction,
but rather at the conceptual level. Each of these languages
provides a certain set of tools and data structures and a
characteristic way (maybe several ways) in which to express
common operations on these data structures. This is the
"model of computation" that I referred to above and as
anyone who has programmed in several different languages
knows, this model is sometimes subtly and sometimes quite
grossly different in different programming languages.

> in fact it limits the things you can write so you can't use
> the full power of lisp.

Yes! That is precisely the point of Boot. That is the main
reason for it's existence. It's just like you cannot use
the full power of assembler language in Fortran.

> 
> boot does not function as a lemma-like level of reasoning.
>

I claim you are wrong. The problem must be that you never
really did "grok" the model of computation that the developers
of Boot had in mind. You are still stuck always thinking in
terms of the way things are done in Lisp.
 
> > Second, I think once the bootstrap step is done, it is ok 
> > to write a compiler for the language in the language itself.
> > So writing a compiler for C in C, a precompiler for C++ in C,
> > one for Boot in Boot, or for Axiom in Axiom is perfectly
> > reasonable. The purist may want to boil everything down to
> > hand-crafted assembly (dare I say machine?) language, 
> 
> .....[snip].....
> 
> swing and a miss, as they say in baseball. 
> 
> the interpreter needs to be:
>    simple to build, 
>    simple to maintain, 
>    simple to extend, 
>    simple to port.
> 
> writing boot in boot forces the system build to be more
> complex.

The additional complexity is irrelevant since it works.
And I don't doubt that it works in the open source version
of Axiom in large part due to your efforts. But of course
it has been working since the early days of Axiom development.

> 
> it forces the maintenance to be harder since you need to
> understand whether the problem is in the boot compiler or
> the interpreter.

That is a normal and expected consequence of using a high
level language. Even when using Lisp one must occasionally
consider whether a problem is due to some coding in Lisp or
perhaps due to some incorrect behaviour on the part of the
implementation of Lisp itself.

> ... 
> writing boot in boot is clever but it makes building,
> maintaining, extending, and porting axiom harder than it
> has to be.

The point of writing boot in boot has nothing to do with
being "clever". It is one of the first goals of most projects
which develop a general purpose language. It shows three very
important things: First that the language has attained a
certain fundamental expressive power and completeness. Second
it also serves as a demonstration of the correctness of a
major part of the language. Third, it represents a transition
from the computational model of the source language to the
higher level computational model of the target language.
>From that point on, it is not longer necessary for the compiler
developer to kept both computational models in mind. The
developers specifically want to free themselves from the old
"mindset".

> 
> yes, this can be fixed but fixing it means that you have to
>   (1) rewrite the boot compiler changes in boot
>   (2) recompile the boot compiler to clisp
>   (3) clip out the changes from the clisp file
>   (4) insert the changes into the bootstrap sections of the 
>       boot compiler
>   (5) rebuild the system
>

All of this can be made completely automatic. But of course
it is only relevant in the rare cases when you need to make
changes to the Boot language itself - something that should
be as rare as making changes to Fortran when writing a
numerical calculation package.
 
> writing boot in boot is clever.

No, it is a logical and normal step in the development of a
new programming language.

> we don't need clever code.

Au contraire! If it wasn't for clever code there would be no
Axiom at all. Human beings are notoriously bad a programming.
We need all the help we can get just to do simple things
correctly. That is why while hardware has increased enormously
in power, software has remained (more or less) at the same
technical level that it reached already 30 years ago. That is
the reason why we are running Axiom now - essentially the
same as it was then - on hardware that must be at least a
1,000,000 times faster and more capable then that on which
Axiom was originally developed.

> 
> boot is NOT a higher level language than lisp. Spad is.



> boot is NOT Spad and is unrelated to Spad.

Simply not true. It is clear that Boot has a direct relation
to Spad as Scot Morrison said.

> boot is ONLY seen by interpreter/compiler developers.

That is true. But where do new interpreter/compiler developers
come from? I have argued that it is very likely and even
desirable that they first be Axiom library developers and
therefore would be very familiar with Spad. They would find
working with Boot much easier than working with the full
power of Lisp.

> boot is used by 1 developer (me) until the documentation
> completes.

That is what motivates me to carry out this "war". You want to
make that 0 developers and for the last 3 years I have been
trying hard instead to increase that number. I want documentation
that makes this possible, not documentation that eliminates
the possibility.

> 
> > > the other "oh by the way" is that while you can seem to
> > > read boot code without knowing lisp (a sort-of pseudo
> > > code syntax) you can't WRITE boot code without knowing
> > > lisp.
> > 
> > Bill page responded:
> > 
> > > Why is that? Surely it is possible to write Spad code
> > > without knowing Lisp? 
> 
> because boot IS a limited version of lisp. for instance, in
> the history mechanism in bookvol5.pamphlet you'll see that
> boot uses RPLAC functions to create a circular list. to write
> this you need to understand that (a) you're using list
> structures, (b) how they are represented (as cons cells)
> and (c) how rplaca/rplacd modifies them. all of that requires
> an understanding of lisp, not boot.

No, that is not true. Lisp is irrelevant. The concept of
a linked list and how to manipulate it is all that is needed.
Linked list structures exactly like this are also available
and used effectively in Spad. So of course there is no
problem understanding and using lists if one only writes in
Boot.

> 
> > > 
> > > Now about this language war ...
> > > [snipped] 
> > > So, *if* I was thinking of such a compromise then there are
> > > some "terms of surrender" that might make this easier:
> > > 
> > > 1) Before any more "lispifying" goes on, *first* move Axiom
> > >    over to ANSI lisp. This will make it possible to write
> > >    lisp that conforms to modern practices and as high level
> > >    as possible. It will also make Axiom less dependent on a
> > >    particular implementation of Lisp and more easily portable.
> 
> so you're suggesting that we modify the boot compiler to generate
> ansi lisp code? yet during the "lispifying" rewrite I AM rewriting
> to ansi lisp.

No, I am not concerned about the specific level of the Lisp
output from Boot. Of course, if we move Axiom to Ansi Lisp,
e.g. the Ansi mode of GCL, then what Boot generates must still
be compilable by the GCL compiler. Can you give some examples
where changes would be required to Boot in order to accomplish
this goal?

>From what I have seen in so far in the new BookVol5 I don't
see much use of high level Ansi lisp constructs. At best the
code looks like it is neutral - sort of the lowest common
denominator. When I referred to using Ansi lisp, above it was
in the context of writing lisp at the highest level of
abstraction possible, e.g. using the kind of destructuring
operations that Jergen Weiss pointed out are fundamental to
Boot.

> 
> > > 
> > > 2) No more just cutting and pasting the generated lisp output
> > >    of Boot into the literate document! This seems pointless
> > >    and a net loss in intelligibility even if it is accompanied
> > >    by a verbal explanation of the code. The original Boot code
> > >    is, as Tim admitted above, significantly more readable as
> > >    "pseudo-code" then what the compiler outputs.
> 
> the cut and paste has to happen. it's the only way to make sure
> the code does what it originally did.

This wouldn't be necessary if you did not insist on re-writing
things in Lisp.

> ... 
> (btw, as a side-effect of the rewrite i've already found and
> corrected two bugs).
>

Surely there are easier and better ways to find bugs than
re-writing something.
 
> 
> believe me, it took a long time with a lot of effort to get
> the system to build without a running axiom system.

Of course. It would also take a long time to get a running
C compiler if you did not already have one.

> 
>  and it ain't pretty. but it should be.
> 
> > > 
> > > 3) Where possible and practical, replace Boot code with Spad
> > >    code as in Dick Jenks original vision instead of Lisp. Surely
> > >    you are not going to argue that Spad is a "dead language"
> > >    written by fewer than 10 surviving programmers? And even if
> > >    that happens to be true ;), that is one thing that we are
> > >    trying hard to change, right?
> 
> wrong level. boot is NOT Spad. the languages are NOT related
> except by authors. they have different design goals.

What you say contradicts what Scot Morrison said.

> 
> and I REALLY don't want to rewrite the boot compiler to make
> it do things like ansi common lisp generation. especially
> if, after having achieved the 3 *terms* I'd would then have
> agreement that I could elide boot from the system. what would
> be the point?

What changes are required of boot if we move Axiom to
Ansi lisp?

> 
> I agree that the compiler needs to be rewritten to do things
> like generate better error messages. 
>   to do THAT level of rewrite requires a deep understanding 
>   of the compiler. 
>     to achieve THAT level of understanding we need to deeply 
>     document how 
>     the compiler works. 
>       to do THAT level of documentation we need to review 
>       each function in detail. 
>         to do THAT detailed level of documentation is my 
>         current goal. 
>         as a side-effect of achieving that level of understanding
>         the code will be in lisp (which is a language I deeply 
>         understand.)

The easiest way to avoid this deeply nested list of things
to do is to use Aldor instead of Spad as was originally
planned for Axiom. The main relevance for Boot, in the short
to medium term would be to support the maintenance of the
Axiom Interpreter. In the longer term perhaps it also could
eventually be re-written in Aldor - but at this point with
the current level of interest in Axiom this seems unlikely
to me.

> ... 
> read the book when it comes out.
> the story turns out ok in the end.
> 

I was kind of hoping that we might all be able to
contribute to the actual *writing* of the book. :)

Regards,
Bill Page.