Re: [Gnu-arch-users] new language, arch, furth, etc.

[Jeremy Shaw]

>   If Tom is listening, could you put into the stack of things to talk 
> about how you see communication with external processes fit into your 
> new language? 

Oh Oh! If we are making requests, can you also talk about document
validation?

[NOTE, the following document is mostly formatted to fit on your
screen, but a few lines exceed 80 character, you may wish to disable
automatic text wrapping in your email client -- M-x
toggle-truncate-lines, or something of that nature.]

I don't like XML much, but one aspect that I do like is the concept
(but not implementation) of DTDs. It seems very useful to say, "here
is a config file and here is a specification for the config file."

For example, imagine an interactive config file editor. If it only
knows the basic syntax, then it can do syntax highlighting and
formatting, and basic syntax checking, but not much else. However, if
you have a DTD-like specification, the editor could give you
interactive help, show you valid choices, make sure all the types are
correct, etc.

Recently I have been playing around with a system LAML, the Lisp
Abstracted Markup Language, which incorporates the idea of scheme as a
configuration language and the use of DTDs:

http://www.cs.auc.dk/~normark/laml/

LAML is designed specifically to work with XML/HTML and DTDs, but many
of the concepts could be applied elsewhere.

Here is how you might do a config file using the LAML stuff. Note that
I am not trying to argue this is the best way, or that it is superior
to rfc2822, or that the tla system will even work at all like this. I
am only presenting *one way* things can be done for informational
purposes. As a matter of fact, in the following example, the
specification can very easily be represented as an rfc822 file, though
I personally think it would be harder to read.

Let's say I wanted to write a tool that would take a specification
file and attempt to merge a bunch of branches into another branch. For
example, let's say I have some sort of multi-merge(?) system where I
put each feature on its own branch, and I merge all the features into
an integration branch for building. Some of the features are big
enough that I split them into sub-features.

So the specification for merging might be represented as a tree
structure:

project--build-dist--version
   ^
   |
   +-- project--upstream--version
   |
   +-- project--debian--version
   |      ^
   |      |
   |      +-- project--debian-fix-1--version
   |
   +-- project--feature-1--version
          ^
          |
          +-- project--feature-1.1--version
          |
          +-- project--feature-1.2--version


In this tree, project--build-dist--version is made by merging in
project--upstream--version, project--debian--version, and
project--feature-1--version.

However, before project--debian--version is merged into
project--build-dist--version, project--debian-fix-1--version is merged
into project--debian--version. A, similarily, feature-1.1 and
feature-1.2 are first merged into feature-1.

To use the LAML I first create a DTD:

<!ELEMENT merge-tree (merge-tree)*>
<!ATTLIST merge-tree revision CDATA #REQUIRED>

I then use laml to convert the DTD, into this s-exp (refered to as an
lsp).

((element "merge-tree" "-" "-" 
        (element-content (seq zero-or-more (name one "merge-tree"))) "") 
        (attribute "merge-tree" (("revision" "CDATA" "#REQUIRED"))))

[ASIDE] In something like tla, I would prefer to just write the
specification in 'lsp' instead of writing a DTD and doing the
conversion. However, in LAML it is assumed you are starting with a
DTD, so the lsp stuff is geared toward machine generation.

I then use laml to generate mirror functions from the lsp. There
mirror functions are written to the file, merge-mirror.scm.

Now I can right a config file that uses the spec:

;; load the mirror functions, this is similar to referencing a DTD 
;; in XML
(load "merge-mirror.scm") 

;; a merge spec for the above example
(merge-tree 'revision "address@hidden/project--build-dist--version"
     (merge-tree 'revision "address@hidden/project--upstream--version")
     (merge-tree 'revision "address@hidden/project--debian--version"
          (merge-tree 'revision 
"address@hidden/project--debian-fix-1--version"))
     (merge-tree 'revision "address@hidden/project--feature-1--version"
          (merge-tree 'revision "address@hidden/project--feature-1.1--version")
          (merge-tree 'revision 
"address@hidden/project--feature-1.2--version")))))

Although this looks like a simple data specification, and can be
treated that way, merge-tree is actually a function -- so I can run
the above code.

Running the above code, generates an AST, which is hard for humans to
read, but easier for programs to parse:

(ast merge-tree ((ast merge-tree () (revision
address@hidden/project--upstream--version) double merge) (ast
merge-tree ((ast merge-tree () (revision
address@hidden/project--debian-fix-1--version) double merge))
(revision address@hidden/project--debian--version) double merge) (ast
merge-tree ((ast merge-tree () (revision
address@hidden/project--feature-1.1--version) double merge) (ast
merge-tree () (revision address@hidden/project--feature-1.2--version)
double merge)) (revision address@hidden/project--feature-1--version)
double merge)) (revision address@hidden/project--build-dist--version)
double merge)

Evaluating the expression, not only generates the above tree, but also
serves as a validation process. For example, the 'revision' attribute
is required, but let's leave one out and see what happens:

  (merge-tree 
              (merge-tree 'revision "address@hidden/project--upstream--version")
              (merge-tree 'revision "address@hidden/project--debian--version"
                          (merge-tree 'revision 
"address@hidden/project--debian-fix-1--version"))
              (merge-tree 'revision "address@hidden/project--feature-1--version"
                          (merge-tree 'revision 
"address@hidden/project--feature-1.1--version")
                          (merge-tree 'revision 
"address@hidden/project--feature-1.2--version")))))

evaluating the expression results is the following message being printed:

"The required attribute revision is not present in the merge-tree
element."

Unfortunately, LAML does not tell us the position of the merge-tree
element in error, but that is a deficiency in LAML, not the concept
itself.

LAML also provides mechanism to convert the spec into XML.

[this snippet missing the boring parts]
(with-input-from-file (vector-ref argv 1)
  (lambda ()
    (display
     (pretty-xml-render
      (eval (read))))
    (newline)))

generates the following for the above spec:

<merge-tree revision="address@hidden/project--build-dist--version">
   <merge-tree 
revision="address@hidden/project--upstream--version"></merge-tree><merge-tree 
revision="address@hidden/project--debian--version">
      <merge-tree 
revision="address@hidden/project--debian-fix-1--version"></merge-tree>
   </merge-tree><merge-tree 
revision="address@hidden/project--feature-1--version">
      <merge-tree 
revision="address@hidden/project--feature-1.1--version"></merge-tree><merge-tree
 revision="address@hidden/project--feature-1.2--version"></merge-tree>
   </merge-tree>
</merge-tree>

With a few extra lines, it could also generate/include the DTD. So,
even though the spec is in some 'arcane' format, it can be quickly
convert into XML/DTD and parsed in python, perl, C, C++, haskell,
scheme, etc.

As I mentioned before, I make no argument that all of these things and
more could not be done in some other language or file format. My goal
in presenting this is to argue *for* having something like a DTD no
matter what format is used to represent the config files.

One might also argue that the above shows a manner in which both camps
might be satisfied[1]. XML is an example of a non-turing complete
language. If tla specified that config files would all be in XML
format, but did not provide a DTD, then writing programs in scheme
that generated valid XML config files would be a pain because each
scheme program would have to try to generate the XML in an ad-hoc
manner.

However, the addition of a DTD + a few hooks, would make it easy to
have config files written in a non-turning complete language, but
write scheme code that generates the config files on-the-fly.

And, no, I am definately *NOT* advocating the use of XML in tla.

Jeremy Shaw.

[1] by both camps, I envision one camp as "full, turing complete
config files", the other camp being, "pre-existing format, non-turing
complete config files". I believe tom is somewhere in the middle.