Re: Dynamic loading progress

[Eli Zaretskii]

> Date: Mon, 16 Feb 2015 10:22:18 -0800
> From: Daniel Colascione <address@hidden>
> CC: address@hidden, address@hidden
> 
> >> The `size' member tells modules how long the emacs_runtime structure
> >> is. (It's better to use size than an explicit version field: this way,
> >> .size = sizeof(struct emacs_runtime) is always correct.)
> > 
> > This approach requires us to change the size each time we change the
> > layout, even if the change itself leaves the size intact.  Using a
> > version field doesn't have this disadvantage.
> 
> Emacs is allowed to pass a new version of the runtime and context
> structures to modules expecting old versions. That can only work if the
> structures are append-only, so `size' is adequate here.

I understand that, but appending members does not necessarily increase
the size of the struct, due to alignment, bit fields, etc.

> > You say "we don't
> > lock ourselves into conservative stack-scanning GC", which I interpret
> > as saying you don't want to rely on stack scanning to avoid a
> > destructive GC in this case.  But if we don't rely on that, where's
> > the guarantee that such emacs_value will survive GC?
> 
> Emacs stores emacs_value values in a local reference table before
> handing them to module code.

That's something that wasn't in your original writeup.  Sounds like a
significant part of the design.

> >> We'll represent all Lisp values as an opaque pointer typedef
> >> emacs_value.
> > 
> > This doesn't play well with --with-wide-int, where a value can be
> > wider than a pointer.  I think we should instead go with intmax_t or
> > inptr_t, whichever is wider on the host.
> 
> emacs_value objects are not literally Lisp_Object values. They're
> indirected through a reference table (either local or global) so that we
> can GC them. A pointer value can address all possible memory locations,
> so it's fine here. (In the wide-int case, the pointed-to table entries
> are wider than pointers, but modules don't need to know that.)

How this will work when some Lisp calls a primitive implemented by a
module.  From the Lisp interpreter POV, it's just a call to a
primitive, so how will this special handling come into existence?
(And why? just to save us from picking up a wider data type?)

> > What about the doc string?
> 
> We'll set that at the lisp level.

I'm not following: what Lisp level?  What I had in mind was a module
that implements a primitive.  Are you saying that every primitive a
module wants to implement will need a Lisp part and a C part?  That
sounds cumbersome, just to save us from passing yet another argument
to make_function.

> >>     emacs_value (*funcall)(
> >>       emacs_env* env,
> >>       emacs_value function,
> >>       int nargs,
> >>       emacs_value args[]);
> > 
> > Shouldn't funcall use emacs_subr?
> 
> Why?

Because otherwise I see no point in having emacs_subr, it's almost
unused.

> >> If Lisp signals or throws, `funcall' returns NULL.
> > 
> > I suggest some other value or indication of that.  NULL is a valid
> > return value, so usurping it for errors might be too harsh.
> 
> No it isn't. Qnil is distinct from NULL in this model because
> emacs_value is not a Lisp_Object in disguise. Qnil is not special here.

I wasn't thinking about Qnil.  I simply don't see why it is a good
idea to forbid funcall from returning NULL in a normal case.

> >> `intern' also does the obvious thing.
> > 
> > Do we need 'unintern' as well?
> 
> Modules can call unintern through Lisp.

Then why provide 'intern'?  It, too, can be called from Lisp.

> >>     emacs_value (*type_of)(
> >>       emacs_env* env,
> >>       emacs_value value);
> >>
> >> Like Lisp type-of: returns a symbol.
> > 
> > What is a "symbol", from the module's C code POV?
> 
> It's an emacs_value. You can compare symbols (by calling `eq'), call
> them as functions, or use them as function arguments. That's sufficient.
> 
> Come to think of it, though, we do need a C-level `eq'.

Why not simply return a C enumeration type?

> Is the Emacs internal encoding stable?

It didn't change since Emacs 23, AFAIK.

But working with the internal representation is not my idea of fun.

> Is consing strings really a bottleneck in the use cases we have in
> mind?

I think we have no idea at this point which use cases we have in mind.

> >  . direct access to buffer text (using buffer-substring means consing
> >    a lot of strings)
> 
> Is that really a problem for the use cases we want to support?

It could be a problem for a module that processes buffer text.

> >  . not sure how will a module "provide" its feature
> 
> It doesn't. You don't require a module: you load it. Most modules will
> come with a small Lisp wrapper, just like Python modules.

So do we _require_ a Lisp wrapper?  Otherwise, how would Emacs know
whether the module is loaded or not?

> > One thing that's inconvenient is the need to drag the environment
> > pointer through all the calls.  Why exactly is that needed?
> 
> Modules have no other way of accessing Emacs internals. Dragging one
> parameter through the system isn't that bad and lets us sanely track
> which module is doing what.

The environment must be known to the module, that's for sure.  But why
does it have to be plugged into every call to every interface
function?

> > Also, the buffer returned by find-file when it returns normally is
> > lost here, isn't it?
> 
> Sure. It's just a tiny example, and the returned buffer itself is still
> GCed.

That's not what bothered me.  I don't understand how will the module
receive that return value in the first place.  An additional argument
to funcall, perhaps?

Thanks.