[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Axiom-developer] Literate Programming -- Knuth interview

From: Eugene Surowitz
Subject: Re: [Axiom-developer] Literate Programming -- Knuth interview
Date: Fri, 18 Nov 2011 12:53:39 -0500
User-agent: Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20111105 Thunderbird/8.0

I was just at BICA2011 and they both record the talks and
are actually creating DVDs along with the collected slides.
(They sold last year's DVDs at this years conference.)
Some are talks are on the web site.

I'm thinking along the line of a "text crawler" complemented
by a relational data base with the capability to call up
the literate pamphlets at a click.  Thus reverse walking
gives a product support and development mechanism.
The net result would be the ability to locate "foobar"
whatever context it is in - code or description.
I am assuming the however well Axiom gets pamphleteer-ed (ouch)
the perversity of human nature will result in individuals
shoehorning and shotgunning in code and comments in when
and where it occurs to them.
I would hope that the easier it is to do literate code cleanly
would discourage that phenomena.

Eugene J. Surowitz

On 11/14/2011 7:12 PM, daly wrote:
On Mon, 2011-11-14 at 14:08 -0500, Eugene Surowitz wrote:
Will your talk and slides be available on the web?

I gave a presentation at the Clojure Conj on Literate
Programming which was fairly well received but there were
no slides. I don't believe the talk was recorded.

I have an article in the Notices of the American Mathematical
Society scheduled to appear in February.

Long term objective excellent; getting there is the rub.
It's ok at the developed and distributed level,
but from the developer's ant eye view the toc and index
are only as good as the choice of structure and
the entry selection.

Indeed, that's a struggle for all authors. However, I have
made the index and cross references for every function and

There is also the headache of what the eyeball missed
and what opinion bias does or does not consider important.
Also texts tend to have concepts expressed as multiword
strings, such as "table of contents".
Pseudo-letters, such as "_" or "~", just add to the mess.

Every function, macro, and variable is introduced with

All of the index entries are hand generated using latex as in:
which will put an entry under "foo" for "bar" and
an entry under "bar" for "foo". This means that it is trivial
to find every place that "bar" is called. This reverse-lookup
usually requires a code walker.

For global variables there are:
which will put an entry in the index for $var.

Exhaustive listing of everything was quite useful
in another book effort.  For a code the size of Axiom,
I guesstimate that a factor of three for the token file size.

Eugene J. Surowitz

On 11/12/2011 9:16 PM, daly wrote:
Well, the question is whether you're looking for a concept
(check table of contents) or a particular function (check
the index). At least that is the long term intention.

On Fri, 2011-11-11 at 11:03 -0500, Eugene Surowitz wrote:
I prefer the fountain pen for novelistic efforts.

The issue that aroused my question has to do with
comments that appear now and then in the list
about knowing what is where in the source code.

The reference to tools really was intended to address
the idea of the utility of a mechanism(s) that could
globally treat the entire source code as text to
answer that question.

Eugene J. Surowitz

On 11/8/2011 4:42 PM, daly wrote:
Tools? The goal of literate programming is communicating from
human to human. It is like writing a novel. All you need is a
working Underwood typewriter and time.

I tend to favor Latex because a lot of math is involved and
Latex prints math well. But I'm presenting a talk about
Literate Programming in North Carolina this week and I plan
to show examples that use XML, html, and video.


On Tue, 2011-11-08 at 15:03 -0500, Eugene Surowitz wrote:
That literate programming is fully justified for Axiom is, well, almost 
But the issue is more how to boost the ability to invert the process
and reverse engineer non-literate code piles into literate documents.

What, in your opinion, would be the most effective type of tool that could
be developed to render the literacy project more tractable?

Eugene J. Surowitz

On 11/8/2011 9:46 AM, address@hidden wrote:
Yet to me, literate programming is certainly the most important thing
that came out of the TeX project. Not only has it enabled me to write
and maintain programs faster and more reliably than ever before, and
been one of my greatest sources of joy since the 1980s -- it has
actually been indispensable at times. Some of my major programs, such
as the MMIX meta-simulator, could not have been written with any other
methodology that I've ever heard of. The complexity was simply too
daunting for my limited brain to handle; without literate programming,
the whole enterprise would have flopped miserably.

If people discover nice ways to use the newfangled multithreaded
machines, I would expect the discovery to come from people who
routinely use literate programming. Literate programming is what you
need to rise above the ordinary level of achievement.

I believe that Axiom's complexity is large enough to demand literate
programming. There are several reasons.

First, computational mathematics requires people to be exceptional at
mathematics and programming. This is a small subset of already small
sets. We might as well add another subset of those who can communicate
their ideas in writing.

Second, there are many design decisions that are necessary to reduce
a mathematical idea to implementation. Some of these design decisions
are mathematically arbitrary (e.g. branch cuts) or computationally
arbitrary (e.g. sparse versus dense) or programatically arbitrary
(e.g. all Spad versus Spad-and-lisp). These design decisions need to
be documented so people who maintain and modify the program know why
the decisions were made. Without this knowledge it would be trivial
to accidently destroy important optimizations.

Third, nobody is an expert in the range of mathematics that Axiom can
and will implement. It is important to present some portions of the
theory associated with domains so people have a clue about the ideas
being encoded. Imagine what would happen if all of the math textbooks
only contained equations but no human-readable text. You might be able
to "read" a calculus textbook but not an infinite group theory textbook.

Fourth, a million line program is too large to put into your head. You
need to have some background on the data structures, control flow, any
special tricks (e.g. funcalls through the symbol-plist), database
design, communication protocols (e.g. between Axiom and Hyperdoc and
Graphics), parsing structures, and a million other details. At best,
the code tells you HOW it does something but not WHY, not what it
depends on, not what depends on it, etc.

Fifth, a program this large and this long-lived will eventually no
longer have the authors around to ask questions. Several of the Axiom
authors are already dead and most are not associated with it anymore.
Some of those authors are the world's expert in their subject matter.
That is a steep hill to climb if you want to understand the code,
especially if you have to debug or modify it.

Sixth, in the 30 year horizon view, we need to pursue a new level of
excellence. As Knuth said:

       "Literate programming is what you need to rise above
        the ordinary level of achievement"

For all these reasons, and more, Axiom needs to be literate.

Tim Daly

Axiom-developer mailing list

Axiom-developer mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]