guile and emacs and elisp, oh my!

[Andy Wingo]

Greetings, all!

My name is Andy, and together with Ludovic Courtès I co-maintain Guile.

I meant to put off writing this note until some months in the future,
but as people seem to be interested in the topic, now is as good a time
as any.

The topic? Emacs and Guile. (Cue cheering and derisions!) I'll be
telling some history here, from the perspective of one who wasn't there,
but I believe I have the essentials right.

Emacs and Guile have a long and lovely history, longer in fact than
Guile's existence itself. Hacking Emacs is the core of GNU -- as you
know, Emacs is alive, it's fungible, it's free. The GNU system grew out
of Emacs and the hack-experience that it represents.

(I have a friend who regularly chides me when I have to reboot and shut
down my Emacs -- it's like I'm killing a friend, he says!)

But when it was decided that GNU should be like a Unix, it was a fall
from grace, in a sense. The system no longer had the Emacs nature. Time
passed, and people looked to see how to spread the Emacs nature from
Emacs itself to the rest of the programs in the GNU system.

One possibility would be to extract elisp from emacs and use it as an
extension language. But given that the language would have to be
extracted anyway, the thought was to use a more modern Lisp dialect,
Scheme. Thus Guile was born, as the GNU extension language.

(Its birth coincided unfortunately with the so-called "Tcl wars", but
Guile was not made because of Tcl. Guile was made out of a need that
existed outside the polemic. For example, Neil Jerram -- recently
retired Guile maintainer, may he hack in peace -- mentioned to me once
that he had contacted Richard quite early on, before the "Tcl wars",
discussing the need for a GNU extension language, as Elisp was to
Emacs.)

But I digress! Tom Lord was the one to make the initial Guile, derived
from SCM as you know, and though his time with Guile was too short, we
are privileged to have traces of his code in Guile, and his ethereal
presence here on this list.

Since then, about 15 years ago now, Guile has had many maintainers. It
was stabilized, and documented, and extended, and hacked, and fixed, but
eventually we realized that Guile's interpreter -- which traversed trees
of expressions to produce its effects and values -- was just too slow.
We needed to do some pre-computation on the source; we needed to compile
it to something.

Given that the U is for ubiquitous (*not* Unique, all you salt-n-pepa
fans), we couldn't do native compilation, not directly anyway -- we
needed something portable. So, with the help of a few hackers, I updated
some work that Keisuke Nishida did on writing a VM and associated
compiler for Guile, and that's what we're going to release hopefully
next month as Guile 2.0.

We've been trying hard to maintain backwards compatibility for Guile,
and we've done well, I think. The move from interpreted to compiled code
has all of the eval-when pitfalls that you all know, but most extensions
just compile fine with 2.0, and practically all correct Scheme code -- a
number of errors were not caught with 1.8 and before -- practically all
of that code runs fine without modification.

If you really want the whole story, I'd advise you to see my
presentation at November's GNU Hackers Meeting in Sweden, which can be
viewed or downloaded here:

  http://wingolog.org/archives/2010/04/02/recent-developments-in-guile



Hi! Still with me? Cool :)

I've said all of this to give some credibility to what I'm going to say
now.

Guile can implement Emacs Lisp better than Emacs can.

We can compile Emacs Lisp to Guile's VM, and make C shims to present a
suitably compatible C interface to the rest of Emacs' C primitives.

No one will notice! Except that after a switch, Emacs would be faster,
more powerful, and have the ability to access all of Guile's facilities
-- the Scheme language, other languages implemented for Guile
(Javascript, Lua, ...), a proper ffi, dynamically loadable libraries, a
module system, the numeric tower (rationals, bignums, etc), Guile's
existing libraries, delimited continuations (!), fast bytevector access,
native threads, etc.

On a language level, I am convinced that not only can Guile implement
Emacs Lisp, but it can provide a coherent story for tight interaction
between e.g. minor modes in Javascript and Scheme and Elisp, etc. More
details will come later, though feel free to bring up specific points.

There have been some suggestions on the list that "Guile people want to
rewrite Emacs in Scheme". Though Scheme does have some facilities to
offer, personally I do not believe in rewriting software. I've failed
enough times, thank-you-very-much -- and if we could retrofit a compiler
into Guile in two years via correctness-preserving transformations,
surely we can replace the implementation of Elisp without causing
problems.

My goal is to make it the obvious decision for the Emacs maintainers to
say, "yes, let's switch to Guile's implementation of elisp, because it
will make Emacs better and have no drawbacks". We're not there yet --
and that's why I wanted to put off this mail -- but we'll get there.

I guess that's all I have to say, for now -- my bandwidth is not with
Emacs right now, as there are enough things to take care of in Guile
itself. But, I'll be responding to this thread, and lurking on the
list, so do get in contact if you have a concern.

In the meantime, happy hacking!

Andy
-- 
http://wingolog.org/