Re: [Savannah-hackers-public] Mailman generates invalid From headers

[Eli Zaretskii]

> Date: Sat, 2 Jan 2021 23:39:06 -0700
> From: Bob Proulx <bob@proulx.com>
> Cc: savannah-hackers-public@gnu.org
> 
> Eli Zaretskii wrote:
> > (If this is the wrong list to post this problem, please tell where to
> > redirect this.)
> 
> I am probably one of the few that continues the distinction that the
> mailing lists are in the "Not Savannah" category.  However I agree
> that there are too many mailing lists to keep track of.
> 
>     https://savannah.gnu.org/maintenance/NotSavannahAdmins/
> 
>     Mailing lists (lists.gnu.org) - the host, mail setup, and Mailman
>     installation are managed by FSF sysadmins. Savannah hackers have
>     limited (non-root) access to mailing lists and archives. All
>     requests for deletion of archived email must go to FSF sysadmin.
> 
> The best place for Mailman issues is mailman AT gnu.org.  All of the
> FSF staff are on that mailing list as well as the volunteer mailing
> list team such as Ineiev and myself too.  So you get the entire group
> there.  But this clearly seems like a bug problem so sending to
> sysadmin and opening an RT ticket seems good too.

Thanks, I've redirected the discussion to those two addresses.

> > Mailman's rewriting of 'From:' addresses in GNU mailing lists
> > sometimes produces broken results.  Here's an example (from the
> > bug-gnu-emacs mailing list):
> > 
> >   From: "gliao.tw--- via "Bug reports for GNU Emacs,
> >    the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> > 
> > (These are 2 actual physical lines.)  As you see, the quotes don't
> > pair up.  The original address seems to be valid, judging by the
> > 'Resent-from:' header:
> > 
> >   Resent-From: "gliao.tw@pm.me" <gliao.tw@pm.me>
> > 
> > Why does this happen?  Can it be fixed?
> 
> Of course this rewriting is required when the sending site declares
> strict DMARC for their domain.  pm.me does this.
> 
>     $ host -t txt _dmarc.pm.me
>     _dmarc.pm.me descriptive text "v=DMARC1; p=quarantine;"
> 
> As far as I know this implemented within Exim on the GNU/FSF systems.
> Ian is the person who will know the details of all of this.  Ian?
> Help!  :-)
> 
> > These unbalanced quotes cause Rmail in Emacs to format the inbox
> > summary badly.  I fixed Rmail, but the result is still sub-optimal,
> > and it would be good to avoid generating such invalid addresses in the
> > first place.
> 
> It does seem that if quotes were embedded in the comment string that
> they would need to be escaped with a backslash.

Thanks, I hope this could be fixed.

I have meanwhile found another problem with this rewriting of From
headers, it has to do with non-ASCII characters in the From address.
Oftentimes the From address includes quoted-printable encoded
non-ASCII characters, as in this example:

  From: =?utf-8?Q?Daniel_Mart=C3=ADn?= <mardani29@yahoo.es>

The message then specifies

  MIME-Version: 1.0
  Content-Type: text/plain

because there are no non-ASCII characters in the message headers and
the body.

When mailman rewrites the From header in this case, it decodes the
quoted-printable part into the corresponding UTF-8 byte sequence, but
leaves the text/plan header intact.  Which is then mishandled by Rmail
in Emacs to present this byte sequence as raw bytes, not as a single
non-ASCII character.  As result I get this in the INBOX buffer
display:

  From: Daniel Mart\303\255n via "Bug reports for GNU Emacs,
   the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>

where the "\303\255" part are two raw bytes displayed instead of the
character í.  What's worse, mailman also puts this decoded address
into the Reply-to header, so replying to this message will use that
garbled address, and now I'm the one to blame for sending a message
with a garbled name of the addressee.

Either mailman should not decode quoted-printable addresses in From,
or it should change the Content-Type accordingly (and add the
appropriate charset= setting there).

Thanks.