Re: SMPT no longer works in Starttls upon upgrading from emacs-22.3-bin-i386 to emacs-23.1-bin-i386 on Windows XP Professional, Service Pack 3.

[Benjamin L. Russell]

X-Draft-From: ("gnu.emacs.gnus" 38318)
References: <80y6owpd14.fsf@Yahoo.com> <804orkb9jc.fsf@Yahoo.com>
        <80fxb489tw.fsf@Yahoo.com> <87ws4cuzdb.fsf@rapitore.luna>
        <80fxay2bn3.fsf@yahoo.com> <871vmhemls.fsf@rapitore.luna>
Gcc: nnml:archive
Newsgroups: gnu.emacs.gnus
Subject: Re: SMPT no longer works in Starttls upon upgrading from 
emacs-22.3-bin-i386 to emacs-23.1-bin-i386 on Windows XP Professional, Service 
Pack 3.
Bcc: 'Benjamin L. Russell' <DekuDekuplex@Yahoo.com>, 'Benjamin Leon Russell 
(Gmail)' <DekuDekuplex@Gmail.com>, 'Benjamin Leon Russell (Yahoo.co.jp)' 
<Dekuplex@Yahoo.co.jp>
From: DekuDekuplex@Yahoo.com (Benjamin L. Russell)
Date: Wed, 09 Sep 2009 15:13:55 +0900
--text follows this line--
Marco Maggi <mrc.mgg@gmail.com> writes:

> "Benjamin L. Russell" wrote:
>> Apparently,  it   is  possible  to   send  e-mail  through
>> gnutils-cli  successfully  using  an  immediate  encrypted
>> bridge
>> [...]
>> However, when I try to  send an EOF to the process through
>> the  Emacs  eshell function  "eshell-send-eof-to-process",
>> for some reason, the TLS handshake fails; [...]
>
> Here is  a copy and paste  of an eshell session  on my Linux
> system (I am doing this  from Italy, whatever this means for
> IP addresses):
>
> | $ gnutls-cli --crlf --starttls --port 587 plus.smtp.mail.yahoo.com
> | Resolving 'plus.smtp.mail.yahoo.com'...
> | Connecting to '76.13.13.48:587'...
> | 
> | - Simple Client Mode:
> | 
> | 220 smtp110.prem.mail.ac4.yahoo.com ESMTP
> | ehlo localhost
> | 250-smtp110.prem.mail.ac4.yahoo.com
> | 250-AUTH LOGIN PLAIN XYMCOOKIE
> | 250-PIPELINING
> | 250 8BITMIME
>
> among the 250 lines there is no STARTTLS capability, so:
>
> | starttls
> | 502 Command not implemented
>
> which means  that the  server does not  support establishing
> the TLS  layer with STARTTLS  through the 587  interface; so
> the only option is to  use port 465 and the immediate bridge
> (SMTP-over-SSL mode).
>
>   If  this is  the  case, it  is  not a  problem with  Gnus,
> Message,  or Emacs  in itself.   Rather it  is a  problem of
> "smtpmail.el" because, AFAICT,  it only supports delayed TLS
> layer creation (STARTTLS mode), as I already reported to the
> Emacs mailing list[1].
>
>   After a  quick inspection of  "smtpmail.el.gz" distributed
> with Emacs 23,  I say that the feature  request has not been
> satisfied; at least, around line 678, I do not see code that
> can  be  used to  build  the  TLS  bridge (send  SIGALRM  or
> whatever) before sending the EHLO greetings.
>
>   So  either you must  use an  external Mail  Delivery Agent
> (which, by the way, is what  I do with the MBFL script).  Is
> my understanding  that you want to use  MSMTP[2] correct?  I
> do not use it, but it seems it can act as MDA for you.
>
>   Again,   first  you   should  make   sure  that   you  can
> successfully send email using a raw command line with MSMTP.

Done.  As you mentioned, it works with port 465, but not with port 587,
as follows:

> $ ./msmtp.exe --host=plus.smtp.mail.yahoo.com 
> --domain=plus.smtp.mail.yahoo.com --port=587 --auth=on --user=dekudekuplex 
> --tls=on --tls-starttls=off --tls-certcheck=off --from=dekudekuplex@yahoo.com 
> dekudekuplex@yahoo.com
> msmtp: TLS handshake failed: error:140770FC:SSL 
> routines:SSL23_GET_SERVER_HELLO:
> unknown protocol
> msmtp: could not send mail
> 
> Benjamin@200706-05 /usr/sbin
> $ ./msmtp.exe --host=plus.smtp.mail.yahoo.com 
> --domain=plus.smtp.mail.yahoo.com --port=465 --auth=on --user=dekudekuplex 
> --tls=on --tls-starttls=off --tls-certcheck=off --from=dekudekuplex@yahoo.com 
> dekudekuplex@yahoo.com
> password for dekudekuplex at plus.smtp.mail.yahoo.com:
> This is a test of msmtp.exe.
> .
> 

Receipt of the sent e-mail message was verified at DekuDekuplex@Yahoo.com.

> If that works, IMHO:
>
> (setq send-mail-function 'message-send-mail-with-sendmail
>       message-send-mail-function 'message-send-mail-with-sendmail)
>
> should work,  with the right configuration,  because you can
> select all the relevant options with the MSMTP configuration
> file.
>
>> Additionally,  here  are  the  relevant portions  from  my
>> msmptrc.txt file
>>
>> defaults
>> tls on
>> tls_certcheck off
>> tls_starttls on
>
> IMHO, reading [4], it should be "tls_starttls off", meaning:
> use TLS with SMTP-over-SSL mode not with STARTTLS mode.

Unfortunately, it still results in the following error message:

> Sending via mail...
> message-send-mail-with-sendmail: Sending...failed with exit value 78

I copied my msmtp configuration file from ~/msmtprc.txt to ~/.msmtprc,
since according to the output of "/usr/sbin/msmtp.exe --version", the
Cygwin version, as opposed to the native Windows version, uses the
latter file as a user configuration file; _viz._:

> $ ./msmtp.exe --version
> msmtp version 1.4.13
> TLS/SSL library: OpenSSL
> Authentication library: GNU SASL
> Supported authentication methods:
> plain cram-md5 digest-md5 external login
> IDN support: enabled
> NLS: disabled
> System configuration file name: /etc/msmtprc
> User configuration file name: /cygdrive/c/home/.msmtprc

> Copyright (C) 2007 Martin Lambers and others.
> This is free software.  You may redistribute copies of it under the terms of
> the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
> There is NO WARRANTY, to the extent permitted by law.

Here are the revised contents of my ~/.msmtprc file (note the changes
from "tls_starttls on" to "tls_starttls off" in both the "defaults" and
"account dekudekuplex" sections, and the change from port 587 to port 465):

> ########################
> defaults
> tls on
> tls_certcheck off
> tls_starttls off

> account dekudekuplex
> host plus.smtp.mail.yahoo.com
> domain plus.smtp.mail.yahoo.com
> tls on
> tls_certcheck off
> tls_starttls off
> auth on
> user dekudekuplex@yahoo.com
> from dekudekuplex@yahoo.com
> password 
> port 465

> logfile c:\home\.msmtplog.txt
> ########################

Additionally, here are the revised relevant portions of my ~/.gnus.el
file:

> (setq send-mail-function 'message-send-mail-with-sendmail)
> 
> (setq message-send-mail-function 'message-send-mail-with-sendmail)
> 
> ; The following settings were adapted from the article "Re: Debugging msmtp 
> in gnus," by Sivaram Neelakantan <nsivaram.net@gmail.com>, dated "Wed, 02 Sep 
> 2009 22:36:12 +0530," on gnu.emacs.gnus
> (setq sendmail-program "c:/cygwin/usr/sbin/msmtp.exe")
> ; (setq sendmail-program "c:/bin/msmtprc/1.4.17/msmtp.exe")
> (setq message-sendmail-extra-arguments '("-a" "dekudekuplex"))
> (setq mail-host-address "yahoo.com")
> (setq smtpmail-debug-info t) ; only to debug problems set to t if needed
> (setq smtpmail-debug-verb t)
> 
> (setq starttls-use-gnutls t)
> 
> (setq starttls-gnutls-program "c:/usr/bin/gnutls-cli.exe")
> 
> ;; Configure outbound mail (SMTP)
> (setq smtpmail-starttls-credentials '(("plus.smtp.mail.yahoo.com" 465 nil 
> nil))
>       smtpmail-smtp-server "plus.smtp.mail.yahoo.com"
>       smtpmail-default-smtp-server "plus.smtp.mail.yahoo.com"
>       send-mail-function 'message-send-mail-with-sendmail
>       message-send-mail-function 'message-send-mail-with-sendmail      
>       smtpmail-smtp-service 465
>       smtpmail-auth-credentials '(("plus.smtp.mail.yahoo.com"
>                                  465
>                                  "dekudekuplex@yahoo.com"
>                                  nil)))

For some reason, "exit value 78" results in Gnus even if msmtp.exe
succeeds in sending an e-mail message at the command line with identical
settings.

>   As a  last resort, if you  know how to use  MSMTP from the
> command line  to send email using  port 465, you  may try to
> adapt  my  library[3]  for   sending  email  with  a  custom
> "message-send-mail-function" function.

That seems to be the only remaining option now.

Should I customize "send-mail-with-mbfl",
"sendmail-mbfl-normalise-message", "sendmail-mbfl-delivery", and
"sendmail-mbfl-post" (see "examples/sendmail-mbfl.el at master from
marcomaggi's mbfl - GitHub" at
http://github.com/marcomaggi/mbfl/blob/master/examples/sendmail-mbfl.el)?

>   MSMTP is  in my  endless "list of  programs to  check out,
> sooner or later", so I am interested in how to make it work,
> too.

If we solve this problem together, we'll both learn how to make it work
in the process, too; that'll be a win-win situation, wouldn't you agree?

-- Benjamin L. Russell

>
> [1] <http://lists.gnu.org/archive/html/bug-gnu-emacs/2009-04/msg00750.html>
> [2] <http://msmtp.sourceforge.net/>
> [3] <http://github.com/marcomaggi/mbfl/blob/master/examples/sendmail-mbfl.el>
> [4] <http://msmtp.sourceforge.net/doc/msmtp.html#Configuration-files>

-- 
Benjamin L. Russell  /   DekuDekuplex at Yahoo dot com
http://dekudekuplex.wordpress.com/
Translator/Interpreter / Mobile:  +011 81 80-3603-6725
"Furuike ya, kawazu tobikomu mizu no oto." -- Matsuo Basho^