bug#27585: segfault when evaluating a file containing only backticks

[Eli Zaretskii]

> From: Steve Kemp <steve@steve.org.uk>
> Date: Thu, 06 Jul 2017 03:46:35 +0000
> 
>   I enjoy reporting bugs when invalid things are tried, because I
>  expect software to be robust, or as much as possible.  If you feed
>  invalid script to gawk, perl, emacs, etc, etc, I expect an error
>  not a segfault.

I enjoy seeing _any_ bugs reported about Emacs, because we want to
make Emacs as robust as possible.  Emacs tries very hard not to crash,
but sometimes the techniques we use cannot work reliably.  As in this
case.  There's a difference between "as robust as possible" and "100%
robust", and expecting not to see segfault _at_all_ from a system with
a full-fledged programming language built into it is unrealistic,
especially when a program deliberately tries to cause Emacs to run out
of memory.

>   I appreciate there are often difficult cases, such as infinite
>  recursion, division by zero, and memory exhaustion, but the idea
>  that ones programming environment is free to crash if your code
>  is wrong is .. unpleasant.

Emacs is not a programming environment.  Emacs is an editor and a
text-processing environment with a built-in extension language.  The
extension language is provided for extending the editor, not for
running arbitrary programs that have no relation to the editor.  You
can, of course, do the latter if you wish, but that is not the main
use case for Emacs, and not the one for which it is optimized.  Making
such a marginal use case more important than it is will tax the much
more important uses of Emacs as an editor and text-processing tool.

Emacs is required not to crash when used in its main roles, or when it
runs one of the packages provided with it.  It should also try very
hard not to crash in other cases as well, but it is not _required_ to
do so, not at all costs anyway.

What that means is that each crash should first be carefully analyzed
in order to understand the underlying reasons and factors which led to
the crash.  Having found the reasons, if there are reasonable ways of
resolving them to prevent the crash, that shall be done.  A crash that
happens during some use case that belongs to the main Emacs use
patterns increases our motivation to seek for a solution, even if the
solutions are hard to find or require complicated changes.  Crashes
outside of the patterns we care about don't produce such an added
motivation, but are still solved if the solution is reasonably
practical and doesn't adversely affects other important uses.

I'm sorry I have to describe all this, which I have no doubt you
understand very well.  It's just that this discussion until now seems
to somehow ignore these simple and clear-cut considerations, without
which I feel the discussion loses its important context, and you
somehow feel that this project is not interested in hearing about
bugs, which is simply not true.

> > >   In the real world?  Nobody.
> >
> > Then why are we discussing this use case?  Let's talk about
> > more practical and interesting cases.
> 
>   I read from this that I should not bother running any more
>  fuzzing, after all if it does result in any crashes they're a result
>  of bogus-coding that would never be hit in the real world.

I suggested nothing of the kind.  Fuzzing can uncover any number of
problems of different nature.  Each one of those should be analyzed
first, before the decision is made whether it should be fixed and with
what priority.  I presume that the above is due to some offense you
took from what I wrote, which is why I thought it was important to
explain what I think should be the way of handling bug reports -- any
bug reports -- submitted against Emacs.

>   That's unfortunate, but I've no particular wish to argue
>  in public.    I will follow your suggestion.

Once again, there was no suggestion from me that fuzzing will never
uncover any bugs we will consider important to fix, or even
unimportant but easy enough to fix.  Each bug report is analyzed on
its own right, and the decision whether to fix it and at what priority
is made independently for each one of them.  It doesn't matter what
method was used to trigger the problem, the analysis and the
conclusion consider the problem itself, not the method used to find
it.

In this case, Emacs already does everything we knew was possible with
modern operating systems (and some of us think we do too much).  Emacs
tries to avoid stack overflow both on the Lisp level and on the C
level, with some non-negligible degree of success.  It is entirely
clear to us that a cleverly constructed Lisp program could circumvent
these defenses and cause a segfault anyway, but Emacs is not supposed
to be a 100% safe environment for running such malicious programs;
crashing for them is IMO way better than some other possible outcomes,
like wiping out the filesystem or wedging the OS kernel.

In sum, I hope you will continue trying to break Emacs and will report
any "successes", so we could improve Emacs in the future.

Thanks.