[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Axiom-developer] Re: [Aldor-l] Literate programming

From: root
Subject: [Axiom-developer] Re: [Aldor-l] Literate programming
Date: Tue, 19 Aug 2008 01:42:01 -0400

>I am getting increasingly interested in literate programming. There are a
>few questionmarks I am turning at the idea of turning the aldor compiler
>sources into literate programs.

I have a long term interest in making Aldor be literate when it
becomes part of Axiom. If the code had been released under the
modified BSD I'd already be working on it.

I can tell you some features of literate programming from experience.
One major effect is that I can program much faster and better. I can't
say why this occurs but my measured output in work (where I use lisp
in a literate format) is 56K lines of code and 6k pages of documentation
over a two year period. It is just so much easier to program this way.

Axiom has 2 large chunks of C code, the graphics and hypertex.

I recently compressed the whole src/graph graphics subdirectory
into a single file (bookvol8.pamphlet) and the whole src/hyper
hypertex directory into a single file (bookvol7.pamphlet). See
Eventually all of Axiom will be in these books.

Two major side-effects of making C programs like hypertex, into a
literate program is that 
  (a) I no longer need local include files and
  (b) programs are single files.

The first effect (a) comes from the fact that 
   #include "foo.h"
can be replaced by 
which has the effect of "inlining" the include file. This has two
other effects. First, you'll discover that it is "standard practice"
to start an include file with noise like:
   #ifndef THISFILE
   #define THISFILE 1
which causes the preprocessor to only "enable" the include file once.
However, if you look closely at the preprocessor output you'll find
that include files are frequently included multiple times. This causes
a lot of scan time in the compile and a lot of useless file I/O. Using
a chunk eliminates both times.

The second effect (b) is caused by the standard practice of tiny C files
that get separately compiled and then linked. These can now be replaced by
inlining the file using chunks, so that
    gcc -c file1.c
    gcc -c file2.c ...
turns into
and the result becomes a single file per user-visible command.

This approach
   (a) eliminates multiple invocations of the compiler
   (b) eliminates multiple accesses to include files
   (c) eliminates preprocessor time from duplicate includes
   (d) eliminates multiple file reads/writes
   (e) reduces link stage processing
   (f) enables compiler block-optimization processing and inlining.
   (g) allows the use of splink, valgrind and other tools trivially

Proper use of hyperlinking in the literate file allows you to walk
all over the PDF using the adobe reader so you can do quick function

The single literate file format eliminates the need for etags or
grep searching in other files. It eliminates the extern lines.
It enables the use of emacs-register tagging to follow jumps.
It trivializes the need for make.

At a conceptual level the literate form gives structure to the
"pile of files". The table of contents can be broken up into logical
sections (e.g. the compiler phases, the platform dependencies, etc)

Additionally you can include graphics. This allows visual explanation
of the file contents. For Axiom graphics I can include images of buttons
near the button code, images of 2D graphs in the 2D section, images
of 3D graphs in the 3D section, examples of the effects of rotation
code as images, etc.

For Axiom hyperdoc, which uses a web-page-link format, I can include 
images of the web pages directly (see bookvol7.1). With the proper
hyperlinks I can walk the static pages of the hyperdoc "web pages"
directly in the PDF file.

In noweb the chunk names are arbitrary. Axiom has looked at 
defining a syntax for chunknames that include functions such as
URLs as in 
   <<http://....>> for external links
   <<xref://....>> for cross-book links
   <<chunk:foo>>   for full chunks
   <<exact:foo>>   for "variable substitution", etc

David Mentre did a simple version of this chunk syntax processor
which lives in the src/doc directory of the Axiom tree. I only
use a single feature of noweb, which is the chunkname mechanism.
Axiom can now process pamphlet files without using noweb at all
since I've added code that understands the format. This allows
me to "read pages" for hypertex directly out of the pamphlet file
rather than using the filesystem. This is much more efficient since
the file generally gets cached in memory and only a single file
handle is used, eliminating the disk I/O.

All of these are simply side-effects of literate programming.  The
primary gain is your ability to organize and control the complexity of
the whole problem. The table of contents mirrors the compiler
structural layout.  Your growing understanding of data structures and
algorithms is documented so you can "recover" that information
quickly.  The AUG information can live next to the code that
implements the feature.  The whole "clever Makefile" problem fades

These, more than any other features, will enable you and anyone who
follows you, to remember what code does and why decisions were
made. If you lose interest in the compiler then at least your
understanding of it will stay with the code.

Aldor needs to become literate since Stephen and Peter won't be
able to answer questions forever. And the answers they have given
are already sinking into the great morass of mailing list history.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]