[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

mockmod.el --- The mockery of a module system for Emacs Lisp

From: Oliver Scholz
Subject: mockmod.el --- The mockery of a module system for Emacs Lisp
Date: Tue, 28 Mar 2006 14:11:42 +0200
User-agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (gnu/linux)

This package is declared to be experimental and I mean it. I post it
here for review and to get comments on the approach used in it, before
I start to test it by rewriting my own packages to use it.

It would help a lot, if I did something similar for the interpreter,
thus making `eval-defun', `eval-buffer' and, of course, edebug DTRT. I
believe it would be possible to make this work intuitively *most of
the time*. But I think this would require to hack `eval' itself, which
is a built-in.

Anyways, I am not going to continue development of this package
without a review by more experienced hackers.

Please email me or crosspost to I am going to
subscribe to the latter for a while.


;;; mockmod.el --- The mockery of a module system for Emacs Lisp

;; Copyright (C) 2005  Oliver Scholz

;; Author: Oliver Scholz <address@hidden>
;; Keywords: lisp

;; This file is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 2, or (at your option)
;; any later version.

;; This file is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to
;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
;; Boston, MA 02110-1301, USA.

;;; Commentary:

;; This package is EXPERIMENTAL and INCOMPLETE.

;; This package implements an ersatz module system.

;; A module system allows to hide function and variable bindings
;; within a module and "export" only those bindings the programmer
;; intentionally declares to be visible from "the outside".

;; This has two advantages: 1. Name clashes are less likely, even if
;; you don't use a package prefix in function and variable names. In
;; fact, you can refrain from using such prefixes at all, thus getting
;; shorter function names and more readable code. 2. You get a clear
;; distinction between functions and variables that are "internal" to
;; the code and those that are supposed to be used by the user or by
;; other programmers. This helps clarifying the code's architecture.

;; Unfortunately, the implementation of Emacs Lisp does not allow for
;; a proper module system. Mockmod.el tries to *simulate* one. This is
;; explained below.

;; How to use it: In order to make your file of Emacs Lisp source code
;; a module, you have to put a module declaration in front of the code
;; (example below). It *must* be the first s-expression in the file.
;; After loading mockmod.el, the Emacs Lisp byte compiler looks for
;; such a module declaration; if it finds one, it compiles the file as
;; a module. The byte compiler will then *in the compiled code* "hide"
;; every function and variable binding not declared to be "exported"
;; in the module declaration.

;; Example:
;; (module lirum-larum

;;   ; Import the modules `foo' and `bar'.
;;   (import (foo "foo.el")
;;           (bar "bar.el"))

;;   ; Import module `baz', exporting again whatever it exports.
;;   (from (baz "baz.el"))

;;   ; Make the variables `schubi' and `dubi' and the function
;;   ; `lalala' available for use in other modules.
;;   (export schubi dubi
;;           (lalala arg1 &optional arg2 &rest args))

;;   ; Make the symbol `lirum-larum' available in Emacs.
;;   (global lirum-larum))

;; The difference between the `global' and `export' clause is that the
;; former will export the binding to the global namespace---i.e. just
;; as without a module. Whereas the `export' clause it meant for
;; `import' to other modules. This is useful for writing libraries
;; that are meant to provide functions for other modules or for
;; splitting large packages into several modules.

;; Note: Importing is not implemented yet.
;; Note: The `from' clause is not implemented yet.

;; IMPORT := import | from
;; EXPORT := export | global
;; OPT-ARG := &optional SYMBOL+
;; REST-ARG := &rest SYMBOL

;; How does it work?

;; In a proper module system the interpreter/compiler resolves symbols
;; identifying a variable or a function in the code to pointers to the
;; appropriate variables (read: "value containers"). A module system
;; tells the interpreter/compiler which symbol<->variable relations
;; are visible within a given module.

;; Mockmod simulates this by rewriting every occurence of function and
;; variable names in the compiled code, if they are defined (with
;; `defvar', `defun' etc.) inside the module. The module name from the
;; module declaration serves as a package prefix for the new name. For
;; instance, if you have an *internal* function named `foo-bar' in a
;; module `baz', Mockmod rewrites its name to `baz::foo-bar' (double
;; colon). If that function is *exported* Mockmod changes its name to
;; `baz:foo-bar' (single colon).

;; Note: At first glance the convention for renaming symbols might
;; resemble some *package systems*. But Mockmod is the mockery of a
;; module system, not the mockery of a package system. A package
;; system works by reading each and every symbol into "packages". The
;; fact that Mockmod does not, has some consequences, for instance
;; with regard to quoted symbols:

;; Emacs Lisp proper does not distinguish between `quote' and
;; `function'. Thus the expressions `(funcall 'function)' and
;; `(funcall #'function)' are equivalent.

;; But `mockmod' does introduce that difference. `mockmod' will
;; resolve function names only within `function' but it will leave any
;; `quote' expression untouched. So you *have* to write, for instance,
;; `(apply #'foo-bar ...)' instead of `(apply 'foo-bar ...)', if
;; `foo-bar' is a function that you defined within your module.

;; Now, a symbol within a `quote' expression is for the
;; interpreter/compiler at first glance just a symbol, i.e. a data
;; type. If the symbol is accessible from outside the body of the
;; function where it is read, `mockmod' is not able to tell whether it
;; is going to be "applied" or "funcalled" or not. Rather than doing
;; some adhoc analysis (like for the simple case of a symbol as the
;; first argument to `apply' or `funcall') `mockmod' forces the
;; programmer to maintain a difference between symbols-as-data and
;; symbols-as-variable-or-function-identifiers.

;; How does it work internally?

;; Mockmod puts a `defadvice' on the function
;; `byte-compile-from-buffer' from `bytecomp.el' which is at the heart
;; of the byte compiler. Called from this advice, mockmod will scan
;; the buffer containing the code for a module declaration. If it
;; finds one, it will then scan the buffer for variable and function
;; definitions and setup two rewrite tables (one for functions names,
;; one for variable names) in the buffer local variables
;; `mockmod-vhash' and `mockmod-fhash'. Also it temporarily redefines
;; some functions from bytecomp.el, like
;; `mockmod-b-c-file-form-defmumble', `byte-compile-normal-call'.
;; These redefinitions do nothing but provide a wrapper to the normal
;; functionality that replaces appropriate symbols in the `form'
;; argument before compilation.

;;; Code:

(eval-when-compile (require 'cl))

(defun mockmod-check-module-declaration (spec &optional name)
  "Check whether SPEC is a valid module declaration.
This raises an error, if SPEC is not valid. If the optional
second argument NAME is non-nil, it should be a symbol specifying
the module name. This function will then raise an error if NAME
does not match the name given in the declaration."

(defun mockmod-get-module-declaration (filename &optional module-name)
  "Return module declaration from fILE."
    (insert-file-contents filename)
    (goto-char (point-min))
    (let ((module-decl (read (current-buffer))))
      (mockmod-check-module-declaration module-decl module-name)

(defun mockmod-module-declaration-name (module-declaration)
  (cadr module-declaration))

;; (defun mockmod-exported-symbols (module-decl filename)
;;   (let ((exports (mockmod-declaration-clause module-decl 'export))
;;         (global-exports (mockmod-declaration-clause module-decl 
;;         (module-name (mockmod-module-declaration-name module-decl))
;;         (make-spec (lambda (sym) (list sym
;;                                        module-name
;;                                        (expand-file-name filename)))))
;;     (cons (mapcar make-spec exports)
;;           (mapcar make-spec global-exports))))

;; (defun mockmod-exported-symbols-from-file (filename module-name)
;;   (mockmod-exported-symbols (mockmod-get-module-declaration
;;                              filename module-name)
;;                             filename))

;; (defun mockmod-imported-symbols (module-decl)
;;   (let ((imports (mockmod-declaration-clause module-decl 'import))
;;         imported-syms imported-global-syms)
;;     (dolist (spec imports)
;;       (let ((res (mockmod-exported-symbols-from-file (cadr spec) (car 
;;         (setq imported-syms (append imported-syms (car res))
;;               imported-global-syms (append imported-global-syms (cdr res)))))
;;     (cons imported-syms imported-global-syms)))

(defun mockmod-declaration-clause (module-decl clause-name)
  "Return the body of all clauses denoted by the symbol CLAUSE-NAME.
If there is more than one such clause, return their bodies merged
into a single list."
  (let (lst)
    (dolist (clause (cddr module-decl) lst)
      (when (eq (car clause) clause-name)
        (setq lst (append lst (cdr clause)))))))

(defvar mockmod-module-prefix nil)

(defvar mockmod-vhash nil)

(defvar mockmod-fhash nil)

(defun mockmod-initialise-hashes (vhash fhash module-spec)
  "Initialise VHASH and FHASH according to MODULE-SPEC.
This fills the hash tables for variable and functions names with
rewrite rules according to the `export', `import' and `global'
clauses from the module declaration. Each key is the unresolved
name. A value in the variable name hash table (VHASH) is just the
resolved name. A value in the function name hash table has the

  ;; FIXME: There is an interface bug here. This function should check
  ;; whether exported functions actually *are* defined in the code.
  (let ((exported (mockmod-declaration-clause module-spec 'export))
        (globals (mockmod-declaration-clause module-spec 'global))
        (module-name (mockmod-module-declaration-name module-spec)))
    (dolist (exp exported)
      (cond ((symbolp exp)
             (puthash exp (intern (format "%s:%s" module-name exp))
            ((listp exp)
             (puthash (car exp)
                      (cons (intern (format "%s:%s" module-name (car exp)))
                            (cdr exp))
            (t (error))))
    (dolist (exp globals)
      (cond ((symbolp exp)
             (puthash exp exp vhash))
            ((listp exp)
             (puthash (car exp)
            (t (error))))))

(defun mockmod-make-internal-name (name)
  "Return the resolved name for an internal function.
NAME is the unresolved name. This function requires the variable
`mockmod-module-prefix' to refer to the proper package prefix."
  (intern (format "%s::%s" mockmod-module-prefix name)))

(defun mockmod-scan-buffer ()
  "Initialize compilation input buffer.
This reads a module declaration, if there is one, and initializes
`mockmod-vhash', `mockmod-fhash' and `mockmod-module-prefix'.
Then it scans the buffer for function and variable definitions
and updates the hashes accordingly."
      (goto-char (point-min))
      ;; Check for module declaration.
      (while (progn (skip-chars-forward " \t\n\l\v")
                                  (looking-at ";"))
                      (forward-line 1))
      (unless (eobp)
        (let ((decl (read (current-buffer))))
          (when (eq (car-safe decl) 'module)
            (mockmod-check-module-declaration decl)
            (set (make-local-variable 'mockmod-module-prefix)
                 (mockmod-module-declaration-name decl))
            (let ((vhash (make-hash-table :test 'eq))
                  (fhash (make-hash-table :test 'eq))
              (mockmod-initialise-hashes vhash fhash decl)

              ;; Scan all forms in the buffer for `defvar's, `defun's etc.
              (while (progn (while (progn (skip-chars-forward " \t\n\l\v")
                                          (looking-at ";"))
                              (forward-line 1))
                            (not (eobp)))
                (setq macroenv
                      (mockmod-scan-form (read (current-buffer)) vhash fhash 
              (set (make-local-variable 'mockmod-vhash) vhash)
              (set (make-local-variable 'mockmod-fhash) fhash)))))))

(defun mockmod-scan-sequence (list vhash fhash macroenv)
  "Scan a list of s-expressions for function or variable definitions.
Return the (possibly updated) macroenvironment."
  (dolist (form list macroenv)
    (setq macroenv (mockmod-scan-form form vhash fhash macroenv))))

(defun mockmod-scan-form (form vhash fhash macroenv)
  "Scan a form for function or variable definitions.
Return the (possibly updated) macroenvironment."
  (let ((form (macroexpand form macroenv)))

    (case (car form)
      (defun (mockmod-process-defun form fhash))
      ((defvar defconst) (mockmod-process-defvar form vhash))
      ((progn prog1 prog2)
       (setq macroenv (mockmod-scan-sequence (cdr form) vhash fhash macroenv)))
      ((let let*)
       (setq macroenv (mockmod-scan-sequence (cddr form) vhash fhash macroenv)))
      ;; What about `if', `cond' ...?
    (if (eq (car form) 'defmacro)
      (cons (cons (cadr form)
                  (cons 'lambda (nthcdr 2 form)))

(defun mockmod-check-lambda (lambda-list pattern)
  "Warn, if LAMBDA-LIST does not match PATTERN.
PATTERN is supposed to be the LAMBDA-LIST as defined in the
  module declaration."

(defun mockmod-process-defun (form fhash)
  "Update FHASH with function or macro name."
  (let ((entry (gethash (cadr form) fhash)))
    (if entry
        (mockmod-check-lambda (car (cddr form)) (cdr entry))
      (puthash (cadr form)
               (cons (mockmod-make-internal-name (cadr form))
                     (car (cddr form)))

(defun mockmod-process-defvar (form vhash)
  "Update VHASH with variable or constant name."
  (let ((entry (gethash (cadr form) vhash)))
    (unless entry
      (puthash (cadr form)
               (mockmod-make-internal-name (cadr form))

;; (defun mockmod-test ()
;;   (interactive)
;;   (set (make-local-variable 'mockmod-module-prefix) 'test)
;;   (save-excursion
;;     (goto-char (point-min))
;;     (mockmod-scan-buffer)
;;     (with-output-to-temp-buffer "*mockmod*"
;;       (let ((print-func (lambda (k v)
;;                           (print k)
;;                           (princ " : ")
;;                           (print v)
;;                           (terpri))))
;;         (maphash print-func mockmod-vhash)
;;         (maphash print-func mockmod-fhash)))))

(defmacro module (&rest args)
  (declare (indent 1))
  (error "Trying to evaluate module declaration"))

(defadvice byte-compile-from-buffer (around mockmod-scan activate)
  (with-current-buffer (ad-get-arg 0)
  (let ((mockmod-old-module-fdef (symbol-function 'module))
        (mockmod-b-c-normal-call (symbol-function 'byte-compile-normal-call))
        (mockmod-b-c-file-form-defmumble (symbol-function 
        (mockmod-b-c-file-form-defvar (symbol-function 
        (mockmod-b-c-variable-ref (symbol-function 'byte-compile-variable-ref))
        (mockmod-b-c-callargs-warn (symbol-function 
        (mockmod-b-c-file-form-defsubst (symbol-function 
        (mockmod-b-c-function-form (symbol-function 
        (progn (fset 'module (cons 'macro (lambda (&rest args) nil)))
               ;; FIXME: the above temporarily redefines `module' to
               ;; be a macro evaluating to nil. This is so, because we
               ;; can't stop the byte compiler from reading the whole
               ;; buffer. Actually a module declaration other than
               ;; being the first form in the buffer should raise an
               ;; error.
               (fset 'byte-compile-normal-call #'mockmod-b-c-normal-call)
               (fset 'byte-compile-file-form-defmumble 
               (fset 'byte-compile-file-form-defvar 
               (fset 'byte-compile-variable-ref #'mockmod-b-c-variable-ref)
               (fset 'byte-compile-callargs-warn #'mockmod-b-c-callargs-warn)
               (fset 'byte-compile-file-form-defsubst 
               (fset 'byte-compile-function-form #'mockmod-b-c-function-form)
      (fset 'module mockmod-old-module-fdef)
      (fset 'byte-compile-normal-call mockmod-b-c-normal-call)
      (fset 'byte-compile-file-form-defmumble mockmod-b-c-file-form-defmumble)
      (fset 'byte-compile-file-form-defvar mockmod-b-c-file-form-defvar)
      (fset 'byte-compile-variable-ref mockmod-b-c-variable-ref)
      (fset 'byte-compile-callargs-warn mockmod-b-c-callargs-warn)
      (fset 'byte-compile-file-form-defsubst mockmod-b-c-file-form-defsubst)
      (fset 'byte-compile-function-form mockmod-b-c-function-form))))

;; The following wrapper functions could be defined with the help of a
;; macro, but I believe it is more readable this way, if a bit
;; tedious.

(defvar mockmod-b-c-normal-call nil)
(defun mockmod-b-c-normal-call (form)
  (let (nname)
    (if (and mockmod-module-prefix
           (setq nname (car (gethash (car-safe form) mockmod-fhash))))
        (funcall mockmod-b-c-normal-call (cons nname (cdr form)))
      (funcall mockmod-b-c-normal-call form))))

(defvar mockmod-b-c-file-form-defmumble nil)
(defun mockmod-b-c-file-form-defmumble (form macrop)
  (let (nname)
    (if (and mockmod-module-prefix
             (setq nname (car (gethash (cadr form) mockmod-fhash))))
        (funcall mockmod-b-c-file-form-defmumble
                 (list* (car form)
                        (cddr form))
      (funcall mockmod-b-c-file-form-defmumble form macrop))))

(defvar mockmod-b-c-file-form-defvar nil)
(defun mockmod-b-c-file-form-defvar (form)
  (let (nname)
    (if (and mockmod-module-prefix
             (setq nname (gethash (cadr form) mockmod-vhash)))
        (funcall mockmod-b-c-file-form-defvar
                 (list* (car form)
                        (cddr form)))
      (funcall mockmod-b-c-file-form-defvar form))))

(defvar mockmod-b-c-variable-ref nil)
(defun mockmod-b-c-variable-ref (base-op var)
  (let (nname)
    (if (and (symbolp var)
             (setq nname (gethash var mockmod-vhash)))
        (funcall mockmod-b-c-variable-ref base-op nname)
      (funcall mockmod-b-c-variable-ref base-op var))))

(defvar mockmod-b-c-callargs-warn nil)
(defun mockmod-b-c-callargs-warn (form)
  (let (nname)
    (if (and mockmod-module-prefix
             (setq nname (car (gethash (car form) mockmod-fhash))))
        (funcall mockmod-b-c-callargs-warn
                 (cons nname (cdr form)))
      (funcall mockmod-b-c-callargs-warn form))))

(defvar mockmod-b-c-file-form-defsubst nil)
(defun mockmod-b-c-file-form-defsubst (form)
  (let (nname)
    (if (and mockmod-module-prefix
             (setq nname (car (gethash (cadr form) mockmod-fhash))))
        (funcall mockmod-b-c-file-form-defsubst
                 (list* (car form)
                        (cddr form)))
      (funcall mockmod-b-c-file-form-defsubst form))))

(defvar mockmod-b-c-function-form nil)
(defun mockmod-b-c-function-form (form)
  (let (nname)
    (if (and (symbolp (cadr form))
             (setq nname (car (gethash (cadr form) mockmod-fhash))))
        (funcall mockmod-b-c-function-form (list 'function nname))
      (funcall mockmod-b-c-function-form form))))

(provide 'mockmod)
;;; mockmod.el ends here

8 Germinal an 214 de la Révolution
Liberté, Egalité, Fraternité!

reply via email to

[Prev in Thread] Current Thread [Next in Thread]