Re: Dynamic loading progress

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: "Stephen J. Turnbull" <address@hidden>

 > I simply don't believe this goal to be practically achievable, if
 > there are no limitations on the interface.

The limitation is obvious: in principle, all Lisp objects are accessed
through Lisp, with the exception of numbers.  Even for strings and
buffers you can call Lisp "coding system" functions.  You may need to
add an "internal char[] buffer" type to serve as source/sink (I'm sure
you already have one for stream I/O, but it probably doesn't have a
formal or stable definition).

 > I also am not sure that people would accept forward incompatibility
 > easily: consider the number of people who use bleeding-edge Org, Gnus,
 > etc. with the last official release of Emacs.  Why would they agree to
 > lose that when part of a module is in C?

For the same reasons they do when a package decides to use some new
feature of Lisp that is forward incompatible.  Mostly, they have no
choice if the package decides to do that.  Upgrade their Emacs or lose
out.  But I don't think you need to worry that the package maintainers
will gratuitously break compatibility for users of C modules.  If
Emacs is unwilling to be disciplined, package maintainers won't use C
modules, or will call through Lisp and otherwise treat external Lisp
objects as opaque.

 > And what about accessing Lisp objects?  Most of them are accessed
 > through C macros, whose definitions change as the internals change.
 > What can you do with those to preserve compatibility across Emacs
 > versions?

Prohibit such change, or add accessor functions with a stable ABI
encapsulating the accessor macros to a table for use by externally
compiled modules loaded at runtime.  A C function call is very cheap
compared to a Lisp funcall.  If this is too expensive the module
probably shouldn't be based on Emacs at all, or it can be moved into
core as has been done traditionally.  It's that simple, both as a
constraint on core development and as a software engineering
technique.  This is not rocket science, and hasn't been for a decade
and a half now.  It's just work, which the core developers can and
should impose on the folks who want the feature.

If you expect you must change some ABI in the future and can't/won't
use the method table approach, tell module writers to access them
*through* Lisp and live with funcall overhead.  I assure you there are
plenty of features that can benefit from C code that can live very
happily even with that restriction.

 > More to the point: I simply don't see any reason for calling Lisp from
 > C.  If a module needs to do something in Lisp, let it do it in Lisp
 > directly, as every external package for Emacs does.  C code should
 > be reserved for providing additional primitives that core Emacs
 > lacks, and that's all.  At least that should be the starting point,
 > IMO.

But you're not the one who wants this feature.  The people who do want
this feature, in both XEmacs (which has more than a decade of
experience with ELLs) and now in Emacs, have accessed, or think they
need to access, Lisp objects from their C code.