bug#20623: XML and HTML files with encoding/charset="utf-8" declaration loose BOM; Coding system is reset from utf-8-with-signature to utf-8 on save

[Stefan Monnier]

> Now reported with "fix this or get removed from the distribution"
> severity at <https://bugs.debian.org/883434>.

I'm curious to see if the OP's "grave" severity settings will stick.
"Grave" is defined in https://www.debian.org/Bugs/Developer#severities as:

    makes the package in question unusable or mostly so, or causes data
    loss, or introduces a security hole allowing access to the accounts
    of users who use the package.

The only part that could arguably apply is "causes data loss", but even
that is stretching the meaning of those words, I think.

This said, we should indeed fix this bug.
Not sure how to Do It Right but least this specific problem should be
fixable with a patch along the lines of the one below (guaranteed 100%
untested).


        Stefan


diff --git a/lisp/international/mule.el b/lisp/international/mule.el
index 019e65b2c6..5c0675aa2f 100644
--- a/lisp/international/mule.el
+++ b/lisp/international/mule.el
@@ -1885,6 +1885,12 @@ auto-coding-alist-lookup
        (setq alist (cdr alist))))
     coding-system))
 
+(defun mule--coding-system-compatible-p (cs new-cs)
+  "Return non-nil if CS is one of the coding-systems described by NEW-CS."
+  (let ((base (coding-system-base cs)))
+    (or (eq base new-cs)
+        (eq base (intern (concat new-cs "-with-signature"))))))
+
 (put 'enable-character-translation 'permanent-local t)
 (put 'enable-character-translation 'safe-local-variable        'booleanp)
 
@@ -2038,8 +2044,12 @@ find-auto-coding
                                (save-excursion
                                  (goto-char (point-min))
                                  (funcall (pop funcs) size)))))
-       (if coding-system
-           (cons coding-system 'auto-coding-functions)))))
+       (and coding-system
+             ;; Don't override utf-8-with-signature with utf-8
+             ;; or latin-1-mac with latin-1 (bug#20623).
+             (not (mule--coding-system-compatible-p
+                   buffer-file-coding-system coding-system))
+            (cons coding-system 'auto-coding-functions)))))
 
 (defun set-auto-coding (filename size)
   "Return coding system for a file FILENAME of which SIZE bytes follow point.