Re: Multithread or multiprocess in emacs??

[Po Lu]

Philip Kaludercic <philipk@posteo.net> writes:

> Ergus <spacibba@aol.com> writes:
>
>> Hi:
>>
>> Very recently I have seen a code in an elpa package that starts async
>> subprocesses; but instead of using make-process the package uses
>> make-thread + process-file. This was to reduce the latency and some
>> lagging (not evident in my system, but apparently it annoyed the
>> package's author).
>>
>> So, my question is now: Has anyone measured the overhead created by
>> make-process in critical parts of the code like flymake checkers, or
>> ispell? and compared with creating new threads?
>>
>> If this is somehow significant maybe we may consider adding a helper
>> thread or thread-pool for some purposes as now the C11 standard has the
>> threads.h header.
>
> Note that C11 threads are controversial[0], but also optional.

Indeed, threads cannot be relied on to exist in Standard C.  Many
compilers don't support C11 anyway, and even less come with a C11
runtime.

Threads are an optional feature in Emacs too.

> Also, I might be mistaken, but shouldn't Pthreads in some form be
> available on platforms that Emacs runs on?  Also, see (elisp) C
> Dialect.

pthreads are not always suitable for complex language runtimes.  But
even that is besides the point.

The big problem with threads is to make an Emacs capable of running more
than one thread at the same time work safely.  Consider this pattern,
which is very common throughout Emacs:

  Lisp_Object cons;
  struct Lisp_String *sp;

  cons = < some cons >;

  CHECK_STRING (XCAR (cons));
  sp = XSTRING (XCAR (cons));

even assuming that reads and writes of Lisp_Object are coherently
propagated between threads (which is not true on some kinds of CPU, and
also --with-wide-int configurations in general), there is still the
problem of `XCAR (cons)' changing to something other than a string
between `CHECK_STRING' and `XSTRING'.

So you will first have to get rid of all of these not-so-constant
subexpressions.

Next, you will have to interlock everything that is reasonable to use
from multiple threads at the same time, such as all of alloc.c,
editfns.c, fileio.c, and so on.  And do so correctly.  Then, you will
have to make everything else report an error when used outside the main
thread.

Finally, you will have to make the resulting Emacs with all of the extra
interlocking overhead run almost as fast as it used to.

  (Not to mention all the Lisp that currently assumes two threads
   can't run at the same time.  Just avoid thinking about that for
   now.)

This work will require a lot of manpower and time!  Just count the
number of years it took to interlock the Unix kernel (which, like Emacs,
previously used set-priority-level to handle the limited amount of
reentrancy present when running on a single CPU with interrupts.)