[Axiom-developer] Algebra bootstrap

[Waldek Hebisch]

I would like to share with you recent progress I made.  Namely, I was
able to bootstrap Axiom algebra using no cached Lisp (that means no
Lisp for algebra files, boot and interpreter are different story).

How doeas it work?  I looked once again at 'Loading' messages appearing
in Axiom trace.  I have noticed that only 263 different thing get
leaded.  Of them there is 199 categories, 44 domains associated
with categories (those domains give default implementations for
some functions) and 20 core domains:

AssociationList Boolean Character DoubleFloat Fraction
IndexedList IndexedOneDimensionalArray IndexedString
IndexedVector Integer List NonNegativeInteger OutputForm
PositiveInteger PrimitiveArray Reference SingleInteger
String Symbol Vector

Main bootstrap problems are due to core domains: if we forget about
Reference the other are all mutually dependent.  Also, they depend
on many categories and it turn many categories depend on core
domains.

AFAICS bootstrap layer used by Tim includes almast all core domains
and categories with associated domains.  Once we have compiled
all categories and domains loaded by Axiom we can proceed essentially
in any order.

I spent some time trying to find pattern deciding wether a file
gets loaded during given compilation or not.  There are obvious
reasons to load categories:  Axiom implements Join as runtime
operation, so in order to compute Join it must first load all
categories involved in Join.  Also, it seems that Axiom loads
categories of arguments of constructors.  However, loading of
domains looked almost random.  After some effort I have tracked
this to the optDeltaEntry in nruncomp.boot.pamphlet: appearently
this functions tries to avoid lookups in domain vectors and redirect
calls to final destination.  To do this optDeltaEntry must have
domain vectors available, so it load corresponding domains.
After disabling optimisation done by optDeltaEntry _no_ domains
are loaded during compilarion.

So, it remains to bootstrap categories.  There is still a little
cyclic dependencies between categories, due to constructor
arguments.  But $bootStrapMode switch resolves this dependency,
and it is enough to topologically sort categories based on
ancestor fields.


The final process looks as follows.  The compiler is modified to
have extra switch $bootstrapDomains, which if set to true disables
optimisation in optDeltaEntry.  In the first pass I use topologically
sorted list of categories and compile in $bootStrapMode.  In the
second pass I use $bootstrapDomains switch and compile categories
plus core domains.  In the thired pass I recompile categories
and core domains in normal mode.  Finally, I compile the rest
of algebra in alphabetical order (in fact, I let make choose the
oreder, but the list I give is sorted alphabetically, and make
processes it in order).

There is still problem of databases: the process I outlined above
uses pre-built databases.  Most of the typechecking uses information
form databases, so in order to modify algebra one has to generate
databases first and only later can use bootstrap procedure.  I made
some progress trying to bootstrap with empty databases.  I am able
to compile categories and about 50 domains in $bootStrapMode starting
from empty databases, but the resulting databases still contain too
little (or possibly incorrect) information, so normal bootstrap using
such databases fails relatively early.

-- 
                              Waldek Hebisch
address@hidden