[Axiom-developer] documentation and the crystal

[root]

Bill,

per our long-lost prior discussion i've been documenting axiom.
the idea of documenting a system like axiom has several facets
as i've come to discover. what it means to document axiom has
a lot to do with why you are looking at the documentation. so
in some way the documentation has to be structured into something
like a large crystal egg with many facets. somehow we've got to
design a documentation structure that will allow the same basic
information to be used in dozens of different ways depending on
why you're looking at it. consider that axiom needs documenting:

of the internal structure of the system
  the data structures
  the logical layers (c, lisp, boot, spad)
  the functional separation (compiler, interpreter, browser)
  the system-dependent (sockets) vs independent (lisp)
of the external structure
  the interpreter
  the compiler
  the graphics
  the hyperdoc
  the openmath
  the documentation
of the algebra structure
  the packages
  the domains
  the categories
of the mathematical structure
  the subjects covered
  the theory underlying the subjects
  categories
of the computational mathematics structure
  intermediate expression swell
  simplification
  math types vs computational types
of the user structure (the book)
  the commands for naviation
  the commands for documentation
  the available math functions
of the programming language
  the compiler syntax
  the compiler semantics
  sequencing, conditionals, looping, file i/o
  domain construction
  categorical tests/constraints
of the testing structure
  the mathematics underlying the tests (CATS, computer algebra test suite)
  the actual tests
  boundary conditions
of the literature
  published algorithms
  published theory
  thesis work
of the program proofs
  underlying theorems and lemmas
  program proof books
  
i'm currently mucking around in the algebra structure. in particular
i've catted all of the algebra together into one file and am "reducing"
it to it's primitive structure. this alone is a daunting task as it
starts out with about a quarter million lines which i've slowly reduced
to about 100k lines so far. i'm doing a topological sort of the algebra
to uncover the actual type hierarchy with the idea that it can be
reduced to a lattice. as you recall this problem was done once before
in order to get axiom to compile from scratch.

getting down to this level of detail for documenting makes it clear
that current systems of documenting are hopelessly weak. somehow we
need to take advantage of the computer to leverage and reuse documentation
in creative ways. if we don't we'll just drown in endless documents. ibm
was famous for delivering shelves worth of documentation which was never
used. barnes and noble has whole bookcases of documents on linux. that
way lies madness.

in fact, documentation is probably the wrong idea. we need somehow to
be able to automatically generate information from some core that
represents the axiom system itself. 

so i'm thinking about a "crystal browser", that is, a browser where you
can gaze into a crystal that surrounds axiom. each facet represents a
generated view of the whole system. trivially we could have a facet
that shows algebra source code. we could also have a facet that shows
the type hierarchy, etc. so it is clear we can create automatic facets
with just the existing code and programs that do structure analysis.

more generally we could construct facets that walk into the pamphlets.
one facet could walk the biblio references, another could extract the
tex, a third could walk index terms to find all references to a single
term (e.g. ideal). particularly interesting would be facets that walk
the semantic structure of the system so you could pick out particular
kinds of ideals or proofs using ideals, etc. certain facets could be
used to impose order by certain metrics (like rainbows in real 
crystals). such rainbow facets could show the type lattice ordered
by layer (ala the structure in the src/algebra/makefile). yet more 
generally is that "literate programs" need to have sufficient structure 
to support the crystal.

in particular, we need to look at some technology that will do some
automated work for us. one that leaps to mind is a semantic network.
new code would be automatically classified into the system based on
several factors (some hand supplied, some derived from the code).

the idea that it is "just documentation" and "just a browser" is 
a weak notion but a good start. in general one would like to use
facets to CONSTRUCT new algebra, new booklets, new proofs, etc.
so both "documentation" and "browser" are the wrong words.

in 30 year computational mathematicians will need to be able to
deal with the complexity of all of the facets of documentation
i mentioned above. we need to construct tools which help organize
the system in ways that a mathematician can effectively use to do
research work.

the general "visual image" is of a large crystal which you can rotate.
every facet gives a different view of the whole of the axiom system.
thus, a "crystal" surrounding axiom.

hope your christmas went well.

t