bug#44307: 27.1; UTF-8 parts transferred as 8bit in multipart messages fail to decode

[Alexandre Duret-Lutz]

Alexandre Duret-Lutz <adl@lrde.epita.fr> writes:
> In my case the (mm-handle-buffer handle) is multibyte.  This
> multibyteness was preserved by mm-copy-to-buffer while creating the
> handle buffer, but a did not check the original source of it, since the
> comment above the macro suggests that having multibyte parts is OK.

I think I understand why the bug cannot be reproduced using nndoc in
emacs 28.  The following patch, which fixes the unibyteness of the nndoc
buffer is not part of emacs 27:

commit 9d0385d7c7adc810dfd06321b783593b7afb3d58
Author: Lars Ingebrigtsen <larsi@gnus.org>
Date:   Fri Aug 21 15:36:45 2020 +0200

    Fix problem with 8bit content-transfer-encoding in nndoc mbox files
    
    * lisp/gnus/nndoc.el (nndoc-possibly-change-buffer): If we're
    reading an mbox file, it may contain messages that use
    content-transfer-encoding 8bit, which means that we have to treat
    the file as a sequence of byte (bug#42951).  This avoids
    double-decoding -- once by Emacs when inserting the mbox into the
    buffer, and once by Gnus when displaying the articles.


So when mm-with-part process the MIME parts of the problematic message
read from the nndoc group, it receives a multibyte buffer in Emacs 27
but a unibyte buffer in Emacs 28. 

However the issue is not restricted to nndoc.

With both emacs 27 & 28 I'm still having the original issue with mail
read using nnmaildir.  So maybe nnmaildir deserves a similar fix?
I'm not quite sure where that would go.  nnmaildir-request-article?

In any way I'm currently running Gnus with the following change:

--- a/lisp/gnus/mm-decode.el
+++ b/lisp/gnus/mm-decode.el
@@ -1264,8 +1264,8 @@ mm-with-part
      (when (and (mm-handle-buffer handle)
                (buffer-name (mm-handle-buffer handle)))
        (with-temp-buffer
-        (mm-disable-multibyte)
         (insert-buffer-substring (mm-handle-buffer handle))
+        (mm-disable-multibyte)
         (mm-decode-content-transfer-encoding
          (mm-handle-encoding handle)
          (mm-handle-media-type handle))

and that seems to solve my problem in both version of Emacs, and in both
nndoc or nnmaildir.

-- 
Alexandre Duret-Lutz