bug#15648: 24.2.50; gnutls SSL connection to IMAP server causes emacs to crash completely

[Eli Zaretskii]

> From: <emacs@kosowsky.org>
> Date: Tue, 12 Nov 2013 10:19:42 -0500
> Cc: emacs@kosowsky.org, tzz@lifelogs.com, 15648@debbugs.gnu.org,
>  Stefan Monnier <monnier@IRO.UMontreal.CA>
> 
> Stefan -- please pay attention to the hostile, unfriendly attitude and
> ultimate arrogance of the Windows Emacs maintainer...

There's no hostility here.  Disagreement, yes; but that's something
different.  As for the other vices, I'll leave it to others to judge
that.

>  > > No, the crux of the problem was and is a bug in the gnutls.el code,
>  > > due to the inconsistency in which magic files are handled.
>  > > 1. file-exists-p is enabled with magic file handling turned on
>  > 
>  > Only because you use cygwin-mount.
> 
> Or any other magic file handler...
> where file-exists-p changes the file path from something not OS
> recognizable to something OS recognizable.

No, cygwin-mount is fundamentally different from most (all?) other
file-handlers.  IMO, it is also a far cry from what the current design
of file-handlers in Emacs assumes.

Let me explain.

First, "magic file names" are IMO really a euphemism.  A much more
accurate term is "remote" or "foreign" file names.  At least this is
the only interpretation I can think of under which the current design
of file-handlers makes sense.  Specifically, Emacs does not want to
know anything about handling such files -- it just calls the handler
and returns the result, without any additional processing.  Emacs does
that consistently in many primitives, and therefore it assumes that a
handler for a class of file names will implement _all_ of the handlers
for those operations that make sense with that class of files.

By contrast, cygwin-mount implements handlers for a very small set (I
think 3 or 4) of file-related operations, and for the rest it turns
around and calls Emacs's original code to do the job.  But Emacs's
internal processing of files assumes without checking that _any_ file
not handled by some handler is a local file that fully abides by the
conventions of the local filesystem.  In particular, the
Windows-specific code in Emacs is _riddled_ with such assumptions.  It
will prepend a drive letter to file names that lack one, convert
forward slashes to backslashes and back at will, downcase file names
without thinking twice, even convert them to 8+3 alias form in some
cases.  Put a "magic" or "semi-magic" file name through that code, and
you cannot trust the result.

Here's a simple example: half way through expand-file-name, Emacs
accesses the home directory of a user, to support "~" in file names.
This is done entirely in C, so if the home directory is in Cygwin
/foo/bar format, the result will be a failure to resolve "~".

Another example is with file-attributes: if someone expects to see
Cygwin-style emulation of Posix mode bits that resembles what Cygwin's
'ls -l' will show, they will be deeply disappointed at best.

Etc., etc. -- there are a lot of use cases where cygwin-mount in its
present shape simply isn't complete enough to pretend to be a
full-fledged file-handler agent that plays by the rules that Emacs
expects.  We can add a little Lisp here and a little more there to fix
some use case or another, but that just obscures and obfuscates the
code, and in a few months or years no one knows why we have this or
that piece of code with some strange effect.

Now, it might be possible to develop cygwin-mount to the degree that
it does work reliably (which will most probably mean either
introduction of much more hooks into Emacs to allow that, or
reimplementation of many of the C portions in Lisp).  If someone
submits patches to do that, it would probably be a welcome addition to
Emacs.  But we are not there yet.

> The same error would occur with ange-ftp and tramp
> should you use that style filename in gnutls-trustfiles.

gnutls.el doesn't make sense with remote file names, I agree.  Your
suggestion to bind the handlers alist to nil is IMO TRT for gnutls (as
well as for many other Emacs packages).

>  > > 3. This causes gnutls.el to crash with a totally incomprehensible
>  > >    error message and an equally incomprehensible return code of '-64'
>  > 
>  > It no longer crashes.
> 
> It doesn't crash emacs... but the routine exist on error without
> human-readable explanation (other than an obscure -64 return value).
> Is this code? Is this user friendly?

Probably not; patches to make that error message more legible are
welcome.

But please don't underestimate the value of the solution that
eliminated the crash: not only Emacs will not crash anymore in that
case, something that Emacs should never do due to bugs on the Lisp
level, but it also corrected a fundamental flaw in the code, whereby a
Lisp object was marked as being a GuTLS connection too early, before
it was completely initialized.  So I think this bug report served
Emacs very well indeed, and thank you for helping us resolve it.

>  > It's not an issue with code quality.  It's simply a consequence of the
>  > fact that Cygwin file names _cannot_ be consistently supported in
>  > native Windows programs.  You can solve perhaps 95% or 99% of that,
>  > but not 100%.  If you are willing to live with those limitations,
>  > please be free.  But it will never be supported all the way, it simply
>  > cannot be.
> 
> The perfect is the enemy of the good enough.

This goes both ways, you know.

> I have offered a patch that substantially improves compatibility
> without any downside other than your own unsubstantiated biases.

The downside you don't see is the additional code it adds to Emacs
that has no other reason but to support cygwin-mount.  There's already
more than enough code in Emacs which no one fully understands, because
it was introduced to "fix" this or that corner case on Windows.  With
almost all the active maintainers coming from the Posix world, I fear
that leaving all this stuff, let alone adding to it, is a slippery
slope to an abyss.

>  > You are entitled to your opinions.  But I respectfully disagree, and
>  > as long as I'm hold responsible for the Windows port of Emacs, I'm
>  > sorry, but my opinions weigh slightly more.
> 
> You are being arrogant. Emacs is about more people than just yourself
> and about more use cases than just you. Such an attitude is
> incongruous with an open source project and discourages both users and
> participants.
> 
> If such an attitude persists, I will regret helping with debugging
> this and taking my precious time to test your c-code fix. I had
> already fixed the problem myself, so with your attitude why would I
> share my insights with you.

I'm sorry that you feel that way.  I tried to explain above why your
help was very important and appreciated beyond the narrow context of
the crash.

>  > Except that Cygwin "magic" cannot be reliably handled that way.  The
>  > difference is that for other magic file names, we have a handling
>  > agent that is outside of Emacs.  By contrast, cygwin-mount pretends to
>  > do that entirely in Emacs Lisp, which cannot work reliably enough.  As
>  > this example shows.
> 
> It will happen with almost any file handler that uses file-exists-p.

I agree that gnutls.el should ignore the file handlers when it tests
for existence of certificate files.  Clearly, having those files on
remote systems makes no sense.

That is not where I was disagreeing with you.  If that is what I
somehow made you understand, I'm sorry for my imperfect wording.

>  > >  >   3. Don't mix Cygwin file names with native Windows programs.
>  > > 
>  > > #3? Where did you come up with that restriction? Is it documented
>  > > somewhere in the Emacs/Elisp manuals and/or in the release notes? Or
>  > > did you invent that restriction out of thin air?
>  > 
>  > It's not an invention.  It's the result of many years using Emacs on
>  > Windows and hacking it.  You are free to ignore that experience, of
>  > course, but I assure you it is valid.
> 
> But why would you preclude this from working.
> Please give me ONE use case where my patch either causes a failure
> elsewhere or where it bogs down the code.

I'm not sure we are talking about the same thing.  Again, I don't
disagree with the change in gnutls.el, to ignore file handlers.

What I was talking about was adding code that is only needed to
support cygwin-mount, like calling expand-file-name, or changing
convert-standard-filename for the same reason.

>  > As long as the solution is plugging cygwin-mount into more places in
>  > Emacs, such a solution will not be acceptable, sorry.
> 
> You are biased against cygwin-mount.

Yes, I am.  I tried to explain above why.  I hope now you understand
my reasons better.