Re: Dynamic modules: MODULE_HANDLE_SIGNALS etc.

[Daniel Colascione]

On 12/23/2015 09:09 AM, Paul Eggert wrote:
> Philipp Stephani wrote:
>>>> We could merely require that any module needing recursion must call a
>>> new stack-size-checking function at each level of recursion, in order to
>>> detect stack overflow.
>>>
>>> That's performance penalty.  I don't think we should incur that, on
>>> behalf of a problem that really should "never" happen.
>>>
>>
>> It's also impossible. Modules will be written in arbitrary languages and
>> make use of arbitrary libraries, some of which might not even know about
>> Emacs's existence. We cannot impose any constraints on module code.
> 
> There's also the issue that other languages may have their own
> assumptions about how stack overflow is detected, assumptions that may
> disagree with Emacs's method or even prevent Emacs's method with
> working.  My own preference also is to rely on the usual VM techniques
> for ordinary C-code modules, and hope that other languages don't get in
> the way. Still, it may not be possible to do that, and we may be forced
> to impose a bit of software overflow-checking on modules implemented via
> recursion, for module authors who care about reliability.
> 
> Does anybody know how C++ and/or Java and/or Python modules detect and
> recover from stack overflow on GNU/Linux? That would be a reasonable
> sanity check here. (And if nobody knows, that's not a good sign....)

Python uses explicit runtime checks, IIRC. HotSpot uses a guard page.
C++ is just the C ABI and is indifferent to stack overflow. Most C-ABI
programs treat stack overflow as another variety of fatal memory error,
and I think that's the right approach. I don't see why Emacs needs to be
special here.

>>>> Also, any module with an unbounded amount of computation should call
>>>> the
>>> equivalent of QUIT every now and then. If the module API doesn't let (or
>>> ideally, require) modules to do that now, it should. Otherwise it's an
>>> Emacs freeze waiting to happen.
>>>
>>> I agree, but I think it's unrelated to stack overflow.  (Though I
>>> think we will need to have a module-specific version of QUIT, as long
>>> as we keep the constraint of not directly longjmping from module
>>> code.)
>>>
>>
>> There's no harm in providing such a functionality (but of course we can't
>> enforce its use).
> 
> We *could* enforce the use by requiring at least one call to QUIT every
> (say) 100 ms, and by doing the equivalent of 3x C-g when the time limit
> is exceeded. That'd be useful even without modules.

That's grossly unacceptable. Individual page faults (over which programs
have little control) can take longer than that.

Besides, modifying all code to fit into Emacs' idiosyncratic model of
stack overflow detection is unreasonable. Modules exist in large part to
call into pre-existing, unmodified libraries.

>> Even if we
>> assume "benign UB" (which is dangerous because UB tends to become more
>> malign as compilers evolve), it will cause internal state to become
>> inconsistent, which is truly disastrous.
> 
> Emacs has been relying on this sort of "benign UB" for years, in areas
> just as disaster-prone as C++ cleanup, and it has worked well enough. I
> don't see why C++ would be any different. Typically C++ cleanup is just
> recovering memory, so we'll have a memory leak; big deal.

Emacs is a monolith and has been written with non-local returns in mind.
Most programs are not. The existing scheme is completely unsafe; you can
do a lot worse than leak. Please stop repeating the false idea that
longjmp from arbitrary points in the program to toplevel is harmless.

> 
>> handle_sigsegv already has several cases where the longjmp is skipped:
>> when
>> in a GC, when the signal is received from the wrong thread, and when it's
>> not guaranteed to be a SO. In those cases Emacs aborts (still
>> autosaving).
> 
> 1. You're right about the GC, but that aspect of GC is considered to be
> a bug (it's called "hard GC error" in the handle_sigsegv comment) and
> should not be precedent for us installing similar bugs elsewhere.
> 
> 2. The wrong-thread check is because we are attempting to detect stack
> overflow only within Emacs (which can occur due to a user program error,
> which should not crash Emacs); stack-overflow errors in other threads
> are considered to be internal errors in Emacs and indeed never should
> happen, so it's OK to treat them as fatal errors (I expect this should
> be true even if the other threads were created by a module).
> 
>> Why can't we do the same if SO is detected while a module function is
>> active?
> 
> It might be reasonable to do that if the stack overflow were entirely
> due to the module -- that'd be like stack overflow in some other thread
> created by the module. But it would not work when Lisp code almost
> overflows the stack, then calls the module code, which overflows; in
> this case Emacs needs to recover rather than aborting.

The _only_ thing Emacs can safely do is abort. The right approach is to
make sure aborting minimizes data loss. There is no reason for Emacs to
be more concerned than other programs about this rare error case.

signature.asc
Description: OpenPGP digital signature