[Axiom-developer] Can you depend on it?

[daly]

It's 5 a.m.  on a sunday morning and I can't sleep because I'm
suffering from flaming asshole syndrome. The person suffering from
this disease feels they know what they are talking about and needs to
enlighten the rest of the world. Today I've got it bad.

I've been programming for nearly 36 years. I'm obsessive about it. I'm
writing a program for work that translates bit streams into mathematics
(in lisp). And for a hobby, I write Axiom programs that translate
mathematics into bit streams (in lisp). When I leave work (whatever
that means, since I work from home and have no fixed hours) I swap
machines and work on Axiom. When I'm not programming, I'm reading
about programming. Of course, I don't read programs, which in some
sense makes me illiterate in my own field. But I do write them. And I
do think about how and why I write them. And how to do it right.
Especially when it concerns Axiom.




In the last 6 years I've given a lot of thought to Axiom. Where can it
go? What can it do? Will it survive? Is it worth keeping? Can people
depend on it?

Ah, that's the question for today...."Can people depend on it?".

And if they can't know the results are correct, what's the point?



I'm re-reading the Mythical Man-Month, which I do every couple years.
And I've recently read "Dreaming in Code", a book about constructing
Chandler, an open-source Personal Information Manager. I watched a
video of destructive testing of the new Boeing aircraft wing. And I'm
remembering Bill Perzley, a guy I used to work for who knew what it
meant to write quality software.

I'll take each of these unrelated components and try to tie them
together.

If you haven't read the Mythical Man-Month I cannot recommend it
enough. It brings up so many good points, like the tarpit problem,
that you need to be able to recognize. Among the many points he makes,
one key "take-away" lesson stands out for me. "Nobody knows how to
schedule software". NOBODY. Even Knuth admits it. So next time your
boss asks you for a schedule don't give him one. Because you don't
know. You can't even guess. Another take-away lesson is that
programming is a lot more than writing code. And that's what concerns
us here. More about this later.




The "Dreaming in Code" book is about a paid project pretending to be
an open source project. It is never going to complete. And it will
fail to do anything worth doing. And no-one can learn from or build
upon the results, good or bad.  It's another long, drawn out software
disaster. I've been there and I've done that. I've written whole
shelves worth of software. Some of it died because the robots that it
drove no longer exist. Some of it died because the company no longer
exists. Some of it died because .... well, because it was crap code,
which nobody but me understood and nobody could maintain. More about
this later.





Which brings me to my third reference point, the destructive testing
of Boeing's new aircraft wing. The video is on youtube and it's really
impressive to watch. The engineering team, dozens of people, is
standing around watching the bending of the wings of a brand new
Boeing aircraft until they snap. The design spec calls for surviving
150 percent of the design load. Both wings finally broke, exactly in
the wing location expected with exactly the kind of failure they
expected.... at 154% of design load. The engineering team went wild
with joy. Except for one guy. He was clearly annoyed and when asked
why he said that they were wrong by about 2.7 percent. (That's the
kind of engineer I want designing airplanes I fly.) Think about that
test. They busted up a perfectly good, multimillion dollar airplane to
test it. And they knew the answer to the test before they ran it.  You
could look up the analysis, read the documentation and rely on the
answer before they ran the test.




Which brings me to my last reference point, Bill Perzley. When I first
started working in robotics I worked on a 3 man research team. Bill
was an old man, in his mid 50s (:-) ) and was the research lead. We
had a robot that could move 200kg at several hundred inches per second
and stop within 1 encoder bit of the target location repeatably. This
was a very big, very fast, and when you made a programming mistake, a
very dangerous machine. The robot had 6 different joints and 16
kilobytes of plated-wire memory. That's 131072 bits. 

We had to teach the robot how to weld the fender of a car using a 30kg
spot welding gun while the fender was stationary. Then we had to
convert the stationary points into moving points so the robot would
weld the fender as it slid by on the assembly line. During our initial
testing a single bit value caused the robot to reach upward while the
weld gun was inside the fender. The robot lifted the weld gun, the
fender, the iron trolley, and 6 meters of steel I-beam up to the
ceiling before we could even think to hit the emergency stop button.
Fast, obedient, and dangerous.

So, Bill spends 4 MONTHS in his office smoking cigars like they were
free, banging away on a 4 function hand calculator. After 4 months he
comes out with a huge pile of A4 paper covered in zeros and ones,
131072 of them to be exact. He hands them to me, along with a pile of
mathematics. He asked me to check it over. And when I'm done to use our
memory store interface (a set of switches to choose a memory location,
set its value, and toggle a "write") to "program" the robot. He had
worked out the final location of every weld as it would appear while
the fender moved past the robot. If his calculations were correct the
robot should spot weld the fender in 24 places in real time
coordinated with the varying speed of the cart along the I-beam.

He had me "program" all 131072 bits into the robot so we could run
this test case. He wanted to see the exact results of his test and the
only way to do that was to stick his head into the wheel well of the
fender, inches from the robot and WELL within the robot's
reach. Needless to say, I was a bit nervous as a single bit mistake in
keying in the "program" could kill Bill before anybody could react.
Being the engineer that he was he stuck his head in the wheel well and
told me to start the robot. I argued with him.... 

ME:   Bill, if this is wrong it could kill you.
BILL: It's right
ME:   Bill, how do you know you didn't make a mistake?"
BILL: I checked it.
ME:   Suppose the math is wrong?
BILL: Did you check it?
ME:   Yes.

ME:   Suppose I made a mistake keying in the program.
BILL: Did you check it?
ME:   Several times.
BILL: Good. Then it works. Start the robot.

I watched the robot swing a weld gun within inches of his cigar all
along the 6 meters. And it worked, just exactly as Bill said it would.
And he KNEW it would work. Why? Because everything he did was
absolutely the best he could make it. This test, like the Boeing wing
test, was a foregone conclusion. He knew the answer before he ever
gave me the program. And I understood everything he did because he
wrote it all down in detail. You could bet your life on the result.



Now, what's this all got to do with Axiom?

Ah, that's the question for today...."Can people depend on it?".

And if they can't know the results are correct, what's the point?


I believe that Axiom could be the foundation for the next few hundred
years of computational mathematics. Oh, not the program we have
now. But the one it could evolve into if we do it right. Axiom is the
only program I've ever worked on that has that potential. The
mathematics doesn't change and, if correctly implemented, Axiom should
always give you an answer you can depend on. In fact, you should be
able to look at the final result of an Axiom computation and KNOW it
is right. And after that time, you and everyone else can depend on it.
Like an aircraft wing, only better because this is mathematics.

But I also find it frustrating because I don't see any Bill Perzley's
in the game. Bill not only learned the mathematics (we both took the
same course from Lou Paul), he understood it. He "programmed" it (we
didn't have anything but calculators and switches for programming but
it was still programming). He wrote it up so *I* could understand it.
And when he wrote programs they worked and *I* could check them. You
could bet your life on his work. And Bill often did.

Is this the level of quality we see in Axiom? Nope. Is this the level
of quality we can achieve? Yes. So, why don't we achieve it? Why don't
we write Axiom software as if our life depended on it? Why are we
still writing software like the Chandler project? Why can't we say
that this WILL work and it WILL give you the correct answer? What does
it take to get Perzley kind of quality?



After much thought I believe that we need to deeply improve the way
Axiom is written. Consider the latest dust-up that just flew by on the
mailing list (this isn't intended to cast blame, just to illustrate).

The issue is an apparent bug that was in the original Axiom
code. Apparently the computation of:

  f:=taylor(1-x^2)
  sin asin f

gave a wrong answer. A "fix" was applied that gives the answer:

  1 - x^2 + O(x^11)

This was considered wrong and a new "fix" was applied giving the
answer:

 1 - 1/4 x^4 - 1/16 x^6 - 7/768 x^8 - 5/3072 x^10 + O(x^11)


Lets apply Bill Perzley's eyes to this bug. I don't know if the
original Axiom answer was correct, the first "fix" was correct, or the
latest "fix" was correct. That's a problem. And it's a very deep
problem. Much deeper than this one bug. Bill's approach to this would
be to start at the bottom, namely subtraction in (1-x^2) and work his
way up, understanding all he could about each piece. At the end of
several months he's announce the fix. And I'd be willing to bet my
life he was right. Because he understood the problem, he understood
the answer, and he documented it so *I* could understand it.

How hard is this? Really, really hard. Because almost nobody knows
about subtraction in Axiom. Nonsense? Well subtraction is a very
widely implemented operator in many domains such as quaternions.  It
occurs everywhere. Is the definition right? Does it work everywhere?
Does axiom implement subtraction properly for all domains?  Once we
feel confident of that we can look at the "1" portion of the initial
expression. What does "1" mean in Axiom? Does it properly implement
the concept of "1" in mathematics?  Can we depend on it?




So that's one of the key questions. "Can we depend on it?". We may be
a domain experts in robotics and implement DHMatrices in Axiom. But did
we write it up in depth so the next person can understand it? 

By the time we've reached the level of documentation of things like
 
  UnivariateTaylorSeries(Expression Integer,x,0)

we, and everyone who reads our documentation, should be able to say
with life-depending certainty that our answer is correct. Someone 30
years from now should be able to read what we wrote and be as certain
as we are that our answer is correct.



Because we're doing mathematics.
Because we're doing it for people to understand.
Because Axiom's life depends on it.

Tim