bug#48137: 27.2; `package-install-file' fails when loading a package file with DOS line endings

[Ioannis Kappas]

On Thu, May 6, 2021 at 4:26 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: Eli Zaretskii <eliz@gnu.org>,  48137@debbugs.gnu.org
> > Date: Thu, 06 May 2021 09:27:38 -0400
> >
> > That's not sufficient, because if we don't decode the file before we
> > call `package-buffer-info` (from `package-install-from-buffer`), then
> > the <foo>-pkg.el file will have incorrect content (e.g. the non-ASCII
> > chars in the description of the package, will be later incorrectly
> > displayed in `list-packages`).
>
> So you are saying the description of the package needs to be decoded
> before using it for list-packages?  That'd be okay; all I care about
> is that the decoded stuff does NOT replace the original raw bytes, but
> instead is used only where decoding is needed.  IOW, decoding should
> either be done on substrings of the original file, and the result
> stored in strings, or the decoded stuff should be placed in a separate
> scratch buffer, which will be used only where decoding is really
> needed.

Is loading with `insert-file-contents' and saving as 'raw-text the
same as copying the raw bytes of the original file?

`hexlify-buffer' in 'hexl uses 'raw-text to display the raw bytes of
an encoded buffer. I always assumed hexl displayed the actual binary
representation of the underlying file.

In which case, having `package-install-file' load the .el package file
metaphorically and modifying `package-unpack' to store 'single files
with 'raw-text should satisfy the requirement? Thus header parsing is
done in the intended coding system, while the end package is a "copy"
of the original.

Example patch:


diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index ecb2573cab..b5fa020179 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -932,7 +932,7 @@ package-unpack
       ('single
        (let ((el-file (expand-file-name (format "%s.el" name) pkg-dir)))
          (make-directory pkg-dir t)
-         (package--write-file-no-coding el-file)))
+         (package--write-file-raw-text el-file)))
       (kind (error "Unknown package kind: %S" kind)))
     (package--make-autoloads-and-stuff pkg-desc pkg-dir)
     ;; Update package-alist.
@@ -1180,9 +1180,9 @@ package-dir-info
 ;; Set of low-level functions for communicating with archives and
 ;; signature checking.

-(defun package--write-file-no-coding (file-name)
+(defun package--write-file-raw-text (file-name)
   "Write file FILE-NAME without encoding using coding system."
-  (let ((buffer-file-coding-system 'no-conversion))
+  (let ((buffer-file-coding-system 'raw-text))
     (write-region (point-min) (point-max) file-name nil 'silent)))

 (declare-function url-http-file-exists-p "url-http" (url))
@@ -2147,7 +2147,9 @@ package-install-file
         (progn
           (setq default-directory file)
           (dired-mode))
-      (insert-file-contents-literally file)
+      (if (string-match "\\.el\\'" file)
+          (insert-file-contents file)
+        (insert-file-contents-literally file))
       (when (string-match "\\.tar\\'" file) (tar-mode)))
     (package-install-from-buffer)))


Btw,
https://www.gnu.org/software/emacs/manual/html_node/elisp/Coding-System-Basics.html
mentions about the 'no-conversion coding system:

  no-conversion (and its alias binary) is equivalent to raw-text-unix:
it specifies no conversion of either character codes or end-of-line.

but since it is -unix, it does do EOL conversions to LF. Should the
above be corrected to something like:

  no-conversion (and its alias binary) is equivalent to raw-text-unix:
it specifies no conversion of character codes but converts
end-of-lines to the unix convention.


Thanks