bug#39884: 27.0.50; Emacs may destroy outgoing email messages during sending

[Rainer Gemulla]

I am composing emails using Emacs message mode. Shortly after switching to Emacs 27, some people complained to me that they could not read some of emails I sent them. I checked in my local email archive: even my archived emails were not readable with Emacs any more. This is a major bug, and for this reason I switched back to Emacs 26. It is my current understanding that the bug occurs when (1) forwarding messages and (2) signing the forwarded messages using GPG. So not everyone seems to be affected. Nevertheless, I experienced this bug on multiple machines (in fact, on all machines that I tried).

The rest of this email explains where the bug arises and how to reproduce it.

Before sending an email, Emacs converts the email text to MIME. This conversion process is performed by a function called mml-to-mime. I found that this function (or it's interaction with signing) is responsible for Emacs 27 destroying emails: the produced MIME can be invalid. This happens mostly when there are multiple mml parts in the original email, which is the case when forwarding an email.

After some playing around, I found a way to reproduce the bug. It's somewhat painful, but the error consistently arises. I first list the steps, then the original email, then its correct MIME encoding, and the incorrect MIME encoding that Emacs may produce. Note that step 3+4 encode the message for the first time (all fine), steps 5-7 add signing, steps 8-9 encode the orginal message again (without signing, now broken).

1. Run: emacs -Q

2. M-x message-mode
3. Clear scratch buffer, paste original message 
4. M-: (mml-to-mime) --> gives CORRECT result

5. Clear scratch buffer, paste original message 

6. Insert a new line "<#secure method=pgpmime mode=sign>" at start of message (after line "--text follows this line--"). This makes Emacs try to sign the mail.
7. M-: (mml-to-mime) --> throws (expected) signer name error

8. Clear scratch buffer, paste original message 

9. M-: (mml-to-mime) 
  --> broken result (first Content-Type after "text follows..." is wrong)

I tried to track down the bug further and found this piece of code in mml-parse-1, line 284:

(setq tag (list 'part '(type . "text/plain"))
      no-markup-p t
      warn t)

During step 9, this statement is executed, but afterwards, the tag variable is not set to the list mentioned in the statement (seriously!). In my case, it had value 'Content-Type: multipart/alternative; boundary="====-=-="' right afterwards (as in the incorrect result below). At this point I gave up, looks like a deeper problem.

ORIGINAL MESSAGE

From: a@b.ce
To: c@d.de
Subject: Test
--text follows this line--
Test

<#mml type=message/rfc822 disposition=inline>
<#multipart type=alternative>
<#part type=text/plain charset="UTF-8" disposition=inline nofile=yes>
Some text.
<#part type=text/html charset="UTF-8" nofile=yes>
Some more text.
<#/multipart>
<#/mml>

CORRECT RESULT

From: a <at> b.ce
To: c <at> d.de
Subject: Test
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
--text follows this line--
--=-=-=
Content-Type: text/plain

Test


--=-=-=
Content-Type: message/rfc822
Content-Disposition: inline

--====-=-=
Content-Disposition: inline
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="====-=-="

Some text.

--====-=-=
Content-Type: text/html; charset=utf-8

Some more text.

--====-=-=--

--=-=-=--

INCORRECT RESULT AFTER STEP 9
From: a <at> b.ce
To: c <at> d.de
Subject: Test
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
--text follows this line--
--=-=-=
Content-Type: multipart/alternative; boundary="====-=-="
Content-Transfer-Encoding: base64

VGV4dAoK
--=-=-=
Content-Type: message/rfc822
Content-Disposition: inline

--====-=-=
Content-Disposition: inline
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="====-=-="

Some text.

--====-=-=
Content-Type: text/html; charset=utf-8

Some more text.

--====-=-=--

--=-=-=--

pgpjPDTU4ksJ4.pgp
Description: PGP signature