Re: Dynamic loading progress

[Stephen Leake]

Eli Zaretskii <address@hidden> writes:

>> From: Stephen Leake <address@hidden>
>> Date: Sat, 14 Feb 2015 19:02:48 -0600
>> 
>> I took a stab at creating emacs_module_api.h; attached.
>> 
>> To do this, I replaced "#include <lisp.h>"
>> in modules/curl/curl.c with "#include <emacs_modules_api.h>", and copied
>> stuff from lisp.h and other headers to emacs_module_api.h until it
>> compiled. Note the FIXMEs; I could not figure out what was going on in
>> some places, so I just kludged it for now.
>
> Thanks.
>
> I started to write comments to that file, but gave up in despair.  My
> overall conclusion from reviewing it is that, if we want the
> tightly-coupled version of modules, we might as well compile them as
> any other C source file in the Emacs tree, i.e. we could simply
> include lisp.h in its entirety, and all the other headers that it
> pulls in.  I see no other workable alternative that would let modules
> use Lisp objects transparently (as opposed to using them as opaque
> objects), call Lisp primitives directly like we do in Emacs sources,
> etc.

I agree.

>> >    This already presents at least 2 problems.  The first is 32- vs
>> >    64-bit issues.  We need some way of checking that a 32-bit module
>> >    isn't loaded by a 64-bit Emacs, and vice versa.
>
>> Good point. That's a special case of "don't load a module compiled for
>> processor X in an Emacs compiled for processor Y". gnu binutils has
>> facilities for that.
>
> Not sure how Binutils could help here.  AFAIK, dlopen and dlsym (and
> their equivalents) don't invoke Binutils on any platform.

I was thinking Emacs could invoke some binutils function to query the
.dll for this info, before calling dlopen.

>> >    The other problem is more serious: Emacs can be built with or
>> >    without --check-lisp-object-type, and with or without --wide-int;
>> >    how will the module know which implementation to use?
>> 
>> Similar check when the module is loaded. Which means we need some
>> metainfo for each module, and a standard function to retrieve it.
>
> That "metainfo", whatever it will be, will also have to support
> compilation of modules: we need to know at compile time what values of
> USE_LSB_TAG, ENABLE_CHECKING, WIDE_EMACS_INT, optimization flags,
> etc. were used when Emacs was built, because those affect the layout
> of objects inside Emacs and also the availability of certain functions
> in the Emacs binary.

Right.

One solution is to only distribute source, and provide an elisp function
'module-make' (suggested by Stefan) that provides command-line options
to a module makefile for all of those settings.

>> >  . Functions to access values of Lisp objects.  We shouldn't rely on C
>> >    macros like XINT and XWINDOW for that, because macros track too
>> >    closely the internals of each object.  So I propose to use
>> >    functions that will be exposed via the API.
>> 
>> If we follow Stefan's suggestion, then either this function approach is
>> not viable, or we need another #ifdef for module vs emacs.
>
> If we don't use the function approach, we can only have modules that
> are tightly coupled with the version of Emacs for which they were
> written and compiled.  We have no hope of binary compatibility with
> other versions of Emacs, except by sheer luck, and even source-level
> compatibility will be a challenge.

Right.

>> > They are currently written as if the code were an integral part of
>> > Emacs on one of the Emacs C files. Going that way means that modules
>> > will have to be recompiled from sources for each Emacs version, and
>> > practically will have to live in the Emacs tree. Maybe this is what we
>> > want, I don't know.
>> 
>> This is the model I had in mind. Since I need max speed with mixed
>> Ada/lisp code, I need tight integration with the core. The people who
>> need that speed for Ada mode will simply have to recompile the module
>> for each Emacs release; they are coders, so that's not a problem.
>
> Being an Ada programmer (or even a C programmer) doesn't necessarily
> mean you know how to fix compile- and link-time problems with Emacs.
> You've just bumped into that yourself.

Yes, and I'm becomming more sympathetic to your point of view :).

> In practice, I think this means maintainers of modules will have to
> adapt their modules to each Emacs version, and perhaps also keep
> several branches, one each for every version they want to support.

Yes, that was my plan. I do that now at the elisp level.

>> Rather than splitting out emacs_module_api.h from lisp.h, we could add
>> __declspec(dllexport) to a subset of the functions in lisp.h; I suspect
>> that will be a small number.
>
> What for?  

Mostly just to get a confirmed list of the functions that curl.c
currently calls. But I guess the link-time errors I started with is good
enough for that.

So let's take a stab at something closer to your approach.

I'm not familiar with the idioms for making an opaque type in C; would
that just be a pointer to void? something like:

typedef void*  Lisp_Object_Ptr;

-- 
-- Stephe