Re: Emacs design and architecture. How about copy-on-write?

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> You mean that at the end of a thread, it will need to write results back
>> to variables "owned" by the calling thread.  Yes.  Either these variables
>> get marked as thread-global (not very attractive), or else we would need
>> to mark specific variables as not to be copied on write.
>
> After the thread terminates, and perhaps also while it still runs.
> E.g., consider some kind of progress-reporting facility.

I think this can be approached similar to buffer-local values - all the
symbols will have default value (global), and per-thread values. When
threads set symbol value normally (for example, via setq), the
per-thread value is modified (and possibly written back to "global"
after the thread terminates). But thread might need to write "global"
directly - something similar to `set-default-toplevel-value' might be
used then.

>> With a thread fetching Gnus articles, there is the additional
>> complication of having several such threads fetching from several servers
>> at once.  Then access to the result variables would need to be locked, to
>> prevent two threads overwriting eachother's results.  But this is so in
>> any multithreading system, no matter how it's done.
>
> The danger is indeed that most of the variables will need to be
> protected by locks.  If we cannot find a way of avoiding that, we are
> back at the current "only one thread at a time" model, and there's
> nothing to gain.

I think that there is a limit to what can be done generically without
affecting the existing Elisp code. Modifying a global variable
asynchronously might need to be coded explicitly on Elisp level by more
granular locks (to be provided as a part of async thread API).

For example, if Gnus is collecting all the articles into a list of
(("server1" . articles) ("server2" . articles)), only Gnus will know
that it is enough to lock cdr of "server1" when fetching articles from
server 1.

In other scenarios, like modifying hash table, it might be possible to
provide thread-safe implementations on C level. Those might or might not
be good enough, depending on the use-case.

>> I envisage each buffer being "owned" by a thread, possibly a special
>> thread just controlling access to the buffer.  Variables like
>> scroll-margin would be the thread's own binding of it.  There are a large
>> number of dynamic variables in redisplay which, in the current Emacs,
>> when bound by a thread would affect Emacs globally.  This c-o-w proposal
>> would fix this problem.
>
> I'm confused: suppose one thread modifies scroll-margin -- does that
> affect the (global) redisplay?  If it does, how will this "solve" the
> problem?  If it doesn't affect redisplay, how _can_ a thread change
> scroll-margin in order to affect redisplay?

IMHO, the only sane way to utilize the existing redisplay is redisplay
lock - only one thread can request redisplay at a time, using its
thread-local state.

>> I don't think several redisplay threads would be a good idea - usually,
>> there is just one screen Emacs is drawing on.
>
> It's one screen, but each window is redrawn separately (on GUI
> terminals).

Technically yes, but AFAIU the code is written assuming single-threaded
execution. Decoupling redisplay of different windows would require
significant non-trivial changes in xdisp.c

>> The c-o-w idea could steer around at least part of the globality.  I
>> think it would be relatively simple to implement (hah!) and wouldn't have
>> a large run-time cost, though of course there would be some.
>
> I think the copy-on-write idea will work only for thread-local
> variables, and we already have those in the form of let-bindings.  The
> important (and the hard) part of this is elsewhere.

Yup. For example, buffer-local symbols are implemented using forward
pointers, overriding buffer object any time we alter (like let-bind) a
buffer-local value.

>From implementation perspective, it might be easier to extend symbol
value slots of symbol object and buffer-local variable slots of buffer
objects to store values for multiple threads + a default value.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>