Re: [Axiom-developer] Re: Pamphlets and LaTex

[C Y]

Stephen Wilson wrote:
> C Y <address@hidden> writes:
> 
>> --- Stephen Wilson <address@hidden> wrote:
>>
>>> What if \chunk{foo} resulted in referencing the Foo domain defined in
>>> another file?
>> I personally didn't consider it desirable to reference chunks outside
>> of a particular file - it makes it that much harder to understand.  If
>> the idea is to tangle something, I don't really like the idea of
>> tangling a source file out of multiple pamphlets.  In my opinion that
>> level of complexity is always going to cause more difficulties than any
>> advantages it might give.  I would be interested in use cases that can
>> demonstrate the benefits of such an approach - just because I can't
>> think of any certainly doesn't mean there aren't any.
> 
> Look at the algebra, for example.  Look at the interrelationships.  Do
> you want to write your algebra for Boolean in the _same_ pamphlet for
> the rest of the system which depends on it?  Do ( untested :)
> 
>      cd ./src/algebra && cat *.pamphlet > algebra.pamphlet
> 
> I expect that is the situation we would have.

No.  What I would do is have Boolean defined where it logically makes sense,
and then have Axiom itself determine inter-relatedness if I were interested
in it.  I might also include a bibtex reference to the pamphlet containing
Boolean if it were of immediate interest to the concept I happened to be
working on at the time.  Chunk level references shouldn't be necessary.


>>>     @
>>>       Introduction.
>>>     <<chunk1>>=
>>>         (code here)
>>>     <<chunk2>>=
>>>         (more code)
>>>     @
>>>        Some documentation
>>>     <<chunk3>>=
>>>          (etc...)
>>>      EOF
>> Erm.  Do we want to support a style like that?  My first thought upon
>> seeing that would be to think "badly formatted, hard to understand."
> 
> Why?

My eye is looking for chunk start -> chunk end.  Here I see chunk start
-> chunk start -> chunk end -> chunk start ...

>           @
>           The following two routines enables one to add integers, and
>           subtract them.
> 
>           <<add>>=
>               (defun add (x y) (+ x y))
>           
>           <<subtract>>=
>               (defun subtract (x y) (- x y))
> 
> I gives you freedom to organize your code in a fashion which might be
> suitable for your problem.  Note that there is absolutely no
> requirement that you must format your code that way.  The point is
> that you can, if needed.

I would find either:

           <<add>>=
               (defun add (x y) (+ x y))
           @

           <<subtract>>=
               (defun subtract (x y) (- x y))
           @

or

           <<add and substract>>=
               (defun add (x y) (+ x y))
               (defun subtract (x y) (- x y))
           @

preferable in such a case, personally - the legality of other options
just invites code that is more difficult to parse.  Just MHO of course.
 As the tool complexity required to handle the alternatives increases,
and/or the speed of processing decreases.  Is it worth the extra complexity?

We really need the guys who wanted the fast cl-web to weigh in here.
Which tradeoff do y'all want?


>>> The state machine hard wires code blocks:
>>>
>>>       <<chunk>>=
>>>           (code)
>>>       @
>> Yes.  Personally I consider this a Good Thing - the visual signature of
>> a chunk is always consistent and unique.  This makes for easier
>> pamphlet editing, in my opinion - the above example would have me
>> wondering if I was in chunk1 or chunk 2.
> 
> Its like as we have in LaTeX, \section{FOO}.  Its a header.

That syntax threw me for a loop when I was learning LaTeX, too ;-).  It
just seems tidier to me to clearly denote the beginning and end of
chunks - it aids the eye in parsing the text and keeps processing simpler.

> And as I
> said, if you want to structure your code in a way which reads more
> easily for you, there is nothing standing in your way.

Certainly.  My concern (as you have correctly noted) is that expanding
the finite state machine is a complex undertaking and most other
alternatives will lose us the speed, which I was told is quite
important.  We need more opinions here guys - speak up!


> I would object to that.  This is very, very, simple syntax.  You can
> parse the expressions involved by hand almost as easily as you could
> specify them in BNF notation -- perhaps easier.

But can we parse it with the speed the finite state machine shows?  I
don't expect a BNF parser generator could do quite as well, but perhaps
it could get close?

> Of course, a BNF parser-generator would be useful for other problems.

I think there may be a couple young efforts floating around out there.
More goodies to add to the list...

 > If one is convinced that latex syntax is the best, then that is fine
> with me.  I really do not care that much about the syntax.  Just dont
> call the file a latex file.  Call it a pamphlet, which must be
> explicitly woven to generate "real" latex.

So if I'm not mistaken, you are concerned that the assumption that LaTeX
can directly handle pamphlet files with no weave step will constrain our
options with pamphlet style later on?  That does make sense.

> Seriously though.  My first response is to look at the pamphlet file
> name itself.  foo.lisp.pamphlet denotes a lisp pamphlet, for example.
> However, I certainly would want to be able to override the defaults
> which a weave or tangle command would assume by convention.  So, yes,
> I would like to have the option to say "<<chunk:boot>>=" or some such.

Right

> There is also a real advantage to be able to make these kinds of
> specifications in the pamphlet file itself.

It makes many things easier.

> Consider how we do it today in make, where a pamphlet contains both
> the boot code and the cached Lisp.  These are foo.boot.pamphlet files.
> Currently, we lift the information that the root chunk <<*>> denotes
> boot code and the root chunk <<clisp>> contains lisp into a the
> makefiles.  Would it not be nice, for example, to have that
> information available in the pamphlet itself and allow cooperation
> between ASDF, say, and a Lisp based weave command to infer the the
> situation itself?  Why code the same information in two places??

I'm not quite sure yet how to handle weave in ASDF.  My guess is once
the pamphlet is woven operation-done-p for any other chunks in that
pamphlet will simply check, see that it's done, and move on.  Language
details shouldn't enter into it for now - maybe later that will some up.

> Note that this is not a pipe-dream.  I am working on real code which
> does this reliably.

Most impressive!

> Consider this.  Someone is creating a new detailed algebra domain in a
> fancy GUI interface which allows then to inspect and modify their
> creation as they work.  This should be a familiar experience for
> someone who codes in lisp or smalltalk or similar.

That's a programming scenario though, which is different again.  When I
think of CAS use I tend to think of the Mathematica notebook - physics
student sits down and runs his calculations.  That's why I think we need
 as many concrete usage scenarios as we can scare up here.  (I know my
thinking runs along a particular set of rails.)

> Our author gets their new domain working.  They click save.  A
> "pamphlet" is written -- a representation of their efforts which can
> be viewed just as easily in dome fancy GUI as it can in an Emacs
> buffer.

OK.

> In the above, where did the pamphlet originate?  I say it was a "live
> object", capable of being modified and inspected by running code, by
> dynamic human interaction.  The representation of a pamphlet as a file
> is just a special case.

I think there are two cases here - pamphlet writing and "end user" CAS
usage.  In the latter case, pamphlets don't even enter into it - they
just want to solve a few equations for a problem they are working on.
For pamphlet creation, I agree interactive tools are a good thing.

>>> All these things have been discussed before, there are many other
>>> options.  I mention these in particular because they are all
>>> candidates for generating LaTeX output, but would ultimately be
>>> inferred from a set of pamphlets.  You cant just run LaTeX on a
>>> pamphlet and obtain that result, because pamphlets are not LaTeX.
>>>
>> Correct.  I would expect to do some interactive work to obtain those
>> documents or at the very least run some special tools (not tangle or
>> weave as such, maybe (prove-all "file.input.pamphlet") or some other
>> command that builds on top of the basic scanner ability.)
> 
> Now your getting it! :)  This is precisely what I am saying.

OK.  I think I'm getting at an assumption I was making unconsciously.
You are proposing to include all forms of documentation associated with
a particular "chunk" of code in a single pamphlet, and depending on the
type of chunk do different processing steps to obtain different output
documents.  This is a departure from the model I was using, which was
one document for research paper, another for API only (no embedded code
chunks, just a general reference to the research paper containing the code.)

Is there a particular reason why we can't simply do something like:

asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdf
asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdf

<<chunkfoo>>[api]=
apiapiapiapiapiapiapi
@

<<chunkfoo>>[whatever]=
whateverwhateverwhatever
@

asdfasdfasdfasdfasdfasdfasdf

<<chunkfoo>>[spad]=
codecodecodecodecode
@

asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdf
asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdf


> How do you define the database?  How is it going to be constructed,
> defined?  You need to write it down somewhere, and in the case of
> axiom, it will be written down in a pamphlet.  May as well let the
> pamphlets communicate the structure.

I had envisioned it working the same way Axiom does now to load
functionality it needs - the "pseudo-database" lookup abilities.  I am
not sure that proofs will be in pamphlets, actually - there is some
question as to whether they are intended primarily as human readable
documents or tools for mechanical verification of correctness.  In the
latter case they will most likely be automatically generated and never
reach pamphlet stage.  More thinking needs to be done here.

> No, you are not missing anything.  Hence the idea of a documentation
> `chunk'.  You will want to specify the API somewhere.  It could appear
> in a research paper as an appendix or nowhere at all.  The very fact
> that it is an option suggests to me that LaTeX is not the proper
> primitive to encapsulate all that data.

Assuming you want all information pertaining to a particular code chunk
in the same pamphlet and in the same location.

> For example, assume we can assign a type to a chunk, as I mentioned
> above.  Then if the chunk type is Spad, feed it to the parser and ask
> not for the AST, but for a list of export/documentation pairs,
> extracted from the traditional +++ notation used in spad.  The +++
> construct can be viewed as a special case of a "documentation chunk".

An interesting possibility.  My only request in that respect would be to
select and stick with a single consistent method for such documentation
per-language.

> However, using just the raw notion of a `chunk' is not enough.  It
> would be like trying to define mathematics using the single axiom "if
> a < b and a > b, then a = b" (well, maybe not quite that extreme :).

I think I'm beginning to see - does the above roughly represent what
you're thinking?

Cheers,
CY