What's the deal with the module system?

[David Kastrup]

Hi,

in the course of seeing how much code can be shared between
define-builtin-markup-command and define-markup-command, the main
difference appears to be scope.

That's not much of a problem: just call the defining command, then
(export ...) the result.  Except that lilypond jiggles with symbols and
procedure properties and similar in some functions.  I quote:

%%;; to be define later, in a closure
#(define-public toplevel-module-define-public! #f)
#(define-public toplevel-module-ref #f)
#(let ((toplevel-module (current-module)))
   (set! toplevel-module-define-public!
         (lambda (symbol value)
           (module-define! toplevel-module symbol value)
           (module-export! toplevel-module (list symbol))))
   (set! toplevel-module-ref
         (lambda (symbol)
           (module-ref toplevel-module symbol))))

#(defmacro-public define-public-toplevel
   (first-arg . rest)
  "Define a public variable or function in the toplevel module:
  (define-public-toplevel variable-name value)
or:
  (define-public-toplevel (function-name . args)
    ..body..)"
  (if (symbol? first-arg)
      ;; (define-public-toplevel symbol value)
      (let ((symbol first-arg)
            (value (car rest)))
        `(toplevel-module-define-public! ',symbol ,value))
      ;; (define-public-toplevel (function-name . args) . body)
      (let ((function-name (car first-arg))
            (arg-list (cdr first-arg))
            (body rest))
        `(toplevel-module-define-public!
          ',function-name
          (let ((proc (lambda ,arg-list
                        ,@body)))
            (set-procedure-property! proc
                                     'name
                                     ',function-name)
            proc)))))

Now the use of these intricate macros bypassing the (well-documented)
guile module system pervades the whole builtin/not-builtin system.

Why does one need a different, manually implemented module system?
Well, the above code (and the quoted sparingly used comments) are the
whole design and implementation document I can find.  At least the code
does not contain references to anything.

Grep for "toplevel" in the Lilypond tree makes clear that it is a
popular term.  It is used in the syntax, for organizing the info
hierarchy, for book structures, to indicate the top parsing level of a
.ly input file.  There is even mentioning of "toplevel scope" for *.ly
files, but the relation to the guile module system, if any, is not
further mentioned.  There are things like a "toplevel music handler",
but this toplevel appears to be rather that of a dynamic execution
hierarchy than of namespaces.

So what's the deal with the builtin definitions, and why can't one just
use them followed by export for defining public functions?

Why is an undocumented private module system employed?  What problems
does it solve that makes it worth scaring prospective contributors away,
even assuming that it _would_ be documented somewhere somewhat obvious
eventually?

I am trying to get into Lilypond, and it is an exercise of frustration
fighting a lot of badly documented code that _affects_ my ability to
code without obvious benefits or relation to the task that I actually
want to be doing.

Being cleverer than the platform one is working on is a recipe for
unmaintainability.  It will also get in the way of using Scheme
compilers, debuggers and similar tools.

So where do I get to know about the design goals and benefits?

Thanks,

-- 
David Kastrup