axiom-developer
[Top][All Lists]

[Axiom-developer] Re: Literate programming and how it scales

 From: Tim Daly Subject: [Axiom-developer] Re: Literate programming and how it scales Date: Fri, 04 Dec 2009 04:20:54 -0500 User-agent: Thunderbird 2.0.0.21 (Windows/20090302)


(actual text is snipped out since this was not public but I want to record


What kind of scaling are you thinking about? Axiom currently has about 20 books that will eventually have all of the source code in literate form. Some of the books are several thousand pages long. There is hyperlinking within the files. Axiom is about 1 million "things of code" and it will all reside in literate files so I think the
technology scales to really large systems.


If you are talking about scaling in terms of humans I think the problem is not one of technology. The problem is that you really can't do a literate work with dozens of authors. Literature requires one, or at most a few, authors in order to make sure that the "style" and "voice" of the documentation is coherent. So I think that many people can work on the system but only a very few are the "editors/primary authors". I don't think there is a good label for this historically, except perhaps the "Chief Programmer"
from Harlan Mills "Chief Programmer Teams". I don't care for that label.


Ultimately literate programming requires a complete understanding of the system. The point of literate programming is to convey this understanding to the reader in a smooth, consistent way. Since programming is involved it is also important that all of the details are explained, but also important that they are introduced in some "motivated" fashion. It is the "motivation" for code snippets that is usually
left out of other documentation forms (e.g. doxygen, javadoc, etc.).


I see TeX as a major strength since most technical papers are written in Latex.
TeX is also a strength for Axiom because it handles mathematical typesetting
very well and Axiom is a mathematical system.


Every literate system will have at least 2 languages, one for programming and one for writing. I find that TeX does everything I need or can be extended with macros
to do what I need. I don't know how to add macros to an HTML system (without
writing in Perl as you do, which is yet a third language)


Axiom is moving away from noweb, although we have used it for years and I can highly recommend it. Some of Axiom's literate files now have "straight latex" macros
for chunks so where you would write
<<noweb chunk name>>=
@

now Axiom allows you to write

\begin{chunk}{Axiom chunk name}
\end{chunk}

This uses straight latex macros which means that your file is pure Latex.

Since the file is pure Latex you no longer need "weave" so half of noweb is gone.

The only other remaining function is "tangle". Axiom is being rewritten

to understand Latex files directly so it can "tangle" the file without using noweb.
The underlying lisp system (GCL) has been modified to include a "tangle"
function directly so Axiom is not needed to extract code.

Thus we will no longer need noweb in the future.

Programmers have been conditioned to believe that they should work on small
files with "include files" to carry information that is shared.


If you look where this idea comes from historically you'll find there was a memory limitation on the PDP 11/40 (I worked on one so I know) that limited a file to about 4K characters. In order to make larger files you needed to invent "include files".
So C programs started the idea that files should be really small.


Then you needed to invent directories since groups of files in a directory made
up a single larger idea.


Then you needed to invent "make" because it becomes impossible to figure out how to compile all the files in all the directories and get all the parts right.

Then you needed to invent "grep" to find functions.

Then you needed to invent IDEs so you could work with the whole project.

Today I have 4G of memory and a 1T of disk space. I can trivially handle

multi-megabyte files, my functions are all in one file, "make" is trivial, "grep"
is an editor "find" command and I don't need an IDE. Latex provides an
automatic cross-reference mechanism that allows me to find all of the places
where a function is called.

Once we stop pretending we work on a PDP 11/40 life gets so much easier

...[snip]... tool description using HTML, Perl, Javascript

So instead of Latex and Lisp, I now need Lisp, HTML, Perl,
and Javascript?

Is adding new languages an improvement? Are HTML files "documents", i.e.,
are they designed to be read as books? Can you control the layout and fonts
that are so necessary to convey certain information?

...[snip]... features of the new tool


Like any author, I keep my ongoing work in the manuscript, which is the literate file itself. There are no auxillary files, no project files, no directory structure, no includes, nothing.

...[snip]...

I'm sure you have a good tool set that most people would find useful.
Unfortunately, the Axiom project is working to "remove the tools" that
are not absolutely required by making Axiom aware of its own file format
(pamphlet, also known as .tex, files)

Tim