Re: Coding system of compressed PO files is not recognized

[Kenichi Handa]

In article <address@hidden>, Richard Stallman <address@hidden> writes:

>     I don't understand what exactly you are asking.  Could you
>     please paraphrase it?

> I want to understand whether you know of a problem in this area
> that has not been fixed.

Yes, the problem is not yet fixed.

>     At least, the current problem is solved if my patch is
>     accepted.

> Are you waiting for someone else to judge your patch?  If so, who is
> expected to judge it, and how long have you been waiting?

I've been waiting for your reply.  You asked me a question
and I replied (about three month ago).  Thus I thought you
were considering whether or not to accept my patch.

> If it has been more than a week, please post your patch here again.

I attached my patch just a week ago on repling to you
(haven't you read it?).  I'll attach it again here.

---
Kenichi Handa
address@hidden

From: Kenichi Handa <address@hidden>
To: address@hidden
CC: address@hidden, address@hidden, address@hidden
In-reply-to: <address@hidden> (message from Richard     Stallman on Tue, 18 Apr 
2006 16:50:31 -0400)
Subject: Re: Coding system of compressed PO files is not recognized
MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya")
Content-Type: text/plain; charset=US-ASCII
Date: Tue, 25 Apr 2006 16:29:30 +0900
X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham     
version=3.0.3

In article <address@hidden>, Richard Stallman <address@hidden> writes:

> I have forgotten about this issue.  Could you ask me again whatever
> question you are waiting for me to answer?

Actually, in the last mail, I answered to your question as this:

----------------------------------------------------------------------
In article <address@hidden>, "Richard M. Stallman" <address@hidden> writes:

>>  Could you give me an example or two?
>>  Which filename handler function does this?
>>  Looking at an example, I can understand the issue.

> We are failing to communicate.  Your patch is a change in
> po-find-charset.  po-find-charset is not a filename handler function.

The patch contains also a change in
po-find-file-coding-system-guts which is called from
po-find-file-coding-system.  And po-find-file-coding-system
is a function registered in file-coding-system-alist and
thus is called from find-operation-coding-system.  I thought
you meant such a function by "filename handler function" in
the current context.

---
Kenichi Handa
address@hidden
----------------------------------------------------------------------

I attach the whole patch I proposed.

We had been discussing about this topic by the subject: "po
file charset via auto-coding-functions".  Please check mails
on that subject.

Here's a brief summary.

For PO file, the function po-find-file-coding-system is
registered in file-coding-system-alist, and that function
checks the file contents to decide a coding system.

But, if a PO file is in an archive file, tar-mode and
arc-mode call find-operation-coding-system with a filename
that doesn't exist in the system, so
po-find-file-coding-system fails.

To solve it, I modified, tar-mode, arc-mode, and po.el as
the attached patch.  In short, I changed tar-mode and
arc-mode to call find-operation-coding-system with
buffer-file-name instead of the actual filename, and change
po.el to visit a buffer by (get-file-buffer filename).

But, if a PO file is compressed, we still have a problem
because jka-compr also calls find-operation-coding-system
(via decode-coding-inserted-region) with non-existing
filename.  For instance, if we are visiting temp.po.gz,
find-operation-coding-system is called with "temp.po", and
at that time, buffer-file-name is "temp.po.gz".  So,
po-find-file-coding-system can't check the contents.

To solve it, I modified jka-compr as the attached patch.  It
binds buffer-file-name temporarily to "temp.po.gz!temp.po"
and calls find-operation-coding-system (via
decode-coding-inserted-region) with buffer-file-name as
filename.  Thus po-find-file-coding-system can visit it by
get-file-buffer.

It seems that this local binding of buffer-file-name is a
little bit fishy part, but I can't think of any other way to
fix the current problem without a drastic change in the
current mechanism of find-operation-coding-system.

While discussing with this problem, another discussion about
making a file-name-handler for tared or archived file
(e.g. for "temp.tar!temp.po" or "temp.zip:temp.po") was
raised, and it seems that you got confused.

---
Kenichi Handa
address@hidden


Index: lisp/arc-mode.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/arc-mode.el,v
retrieving revision 1.72
diff -c -r1.72 arc-mode.el
*** lisp/arc-mode.el    8 Mar 2006 01:57:59 -0000       1.72
--- lisp/arc-mode.el    25 Apr 2006 06:42:40 -0000
***************
*** 873,879 ****
               (let ((file-name-handler-alist
                      '(("" . archive-file-name-handler))))
                 (car (find-operation-coding-system 'insert-file-contents
!                                                   filename t))))))
        (if (and (not coding-system-for-read)
               (not enable-multibyte-characters))
          (setq coding
--- 873,879 ----
               (let ((file-name-handler-alist
                      '(("" . archive-file-name-handler))))
                 (car (find-operation-coding-system 'insert-file-contents
!                                                   buffer-file-name t))))))
        (if (and (not coding-system-for-read)
               (not enable-multibyte-characters))
          (setq coding
Index: lisp/tar-mode.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/tar-mode.el,v
retrieving revision 1.105
diff -c -r1.105 tar-mode.el
*** lisp/tar-mode.el    6 Feb 2006 14:33:35 -0000       1.105
--- lisp/tar-mode.el    25 Apr 2006 06:42:41 -0000
***************
*** 736,742 ****
                                    (funcall set-auto-coding-function
                                             name (- (point-max) (point)))))
                             (car (find-operation-coding-system
!                                  'insert-file-contents name t))))
                        (multibyte enable-multibyte-characters)
                        (detected (detect-coding-region
                                   (point-min)
--- 736,742 ----
                                    (funcall set-auto-coding-function
                                             name (- (point-max) (point)))))
                             (car (find-operation-coding-system
!                                  'insert-file-contents buffer-file-name t))))
                        (multibyte enable-multibyte-characters)
                        (detected (detect-coding-region
                                   (point-min)
Index: lisp/textmodes/po.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/textmodes/po.el,v
retrieving revision 1.13
diff -c -r1.13 po.el
*** lisp/textmodes/po.el        6 Feb 2006 12:12:26 -0000       1.13
--- lisp/textmodes/po.el        25 Apr 2006 06:42:41 -0000
***************
*** 44,55 ****
    "Return PO charset value for FILENAME."
    (let ((charset-regexp
         "^\"Content-Type:[ \t]*text/plain;[ \t]*charset=\\(.*\\)\\\\n\"")
        (short-read nil))
      ;; Try the first 4096 bytes.  In case we cannot find the charset value
      ;; within the first 4096 bytes (the PO file might start with a long
      ;; comment) try the next 4096 bytes repeatedly until we'll know for sure
      ;; we've checked the empty header entry entirely.
!     (while (not (or short-read (re-search-forward "^msgid" nil t)))
        (save-excursion
          (goto-char (point-max))
        (let ((pair (insert-file-contents-literally filename nil
--- 44,59 ----
    "Return PO charset value for FILENAME."
    (let ((charset-regexp
         "^\"Content-Type:[ \t]*text/plain;[ \t]*charset=\\(.*\\)\\\\n\"")
+       (buf (get-file-buffer filename))
        (short-read nil))
+     (when buf
+       (set-buffer buf)
+       (goto-char (point-min)))
      ;; Try the first 4096 bytes.  In case we cannot find the charset value
      ;; within the first 4096 bytes (the PO file might start with a long
      ;; comment) try the next 4096 bytes repeatedly until we'll know for sure
      ;; we've checked the empty header entry entirely.
!     (while (not (or short-read (re-search-forward "^msgid" nil t) buf))
        (save-excursion
          (goto-char (point-max))
        (let ((pair (insert-file-contents-literally filename nil
***************
*** 57,63 ****
                                                    (1- (+ (point) 4096)))))
          (setq short-read (< (nth 1 pair) 4096)))))
      (cond ((re-search-forward charset-regexp nil t) (match-string 1))
!         (short-read nil)
          ;; We've found the first msgid; maybe, only a part of the msgstr
          ;; value was loaded.  Load the next 1024 bytes; if charset still
          ;; isn't available, give up.
--- 61,67 ----
                                                    (1- (+ (point) 4096)))))
          (setq short-read (< (nth 1 pair) 4096)))))
      (cond ((re-search-forward charset-regexp nil t) (match-string 1))
!         ((or short-read buf) nil)
          ;; We've found the first msgid; maybe, only a part of the msgstr
          ;; value was loaded.  Load the next 1024 bytes; if charset still
          ;; isn't available, give up.
***************
*** 74,80 ****
  Do so according to FILENAME's declared charset."
    (and
     (eq operation 'insert-file-contents)
!    (file-exists-p filename)
     (with-temp-buffer
       (let* ((coding-system-for-read 'no-conversion)
            (charset (or (po-find-charset filename) "ascii"))
--- 78,84 ----
  Do so according to FILENAME's declared charset."
    (and
     (eq operation 'insert-file-contents)
!    (or (get-file-buffer filename) (file-exists-p filename))
     (with-temp-buffer
       (let* ((coding-system-for-read 'no-conversion)
            (charset (or (po-find-charset filename) "ascii"))
Index: lisp/jka-compr.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/jka-compr.el,v
retrieving revision 1.90
diff -c -r1.90 jka-compr.el
*** lisp/jka-compr.el   27 Feb 2006 01:51:17 -0000      1.90
--- lisp/jka-compr.el   25 Apr 2006 06:42:41 -0000
***************
*** 475,484 ****
             (delete-file local-copy)))
  
          (unless notfound
            (decode-coding-inserted-region
             (point) (+ (point) size)
!            (jka-compr-byte-compiler-base-file-name file)
!            visit beg end replace))
  
          (and
           visit
--- 475,488 ----
             (delete-file local-copy)))
  
          (unless notfound
+           (let ((buffer-file-name
+                  (concat file "!"
+                          (jka-compr-byte-compiler-base-file-name file))))
+ 
            (decode-coding-inserted-region
             (point) (+ (point) size)
!            buffer-file-name
!            visit beg end replace)))
  
          (and
           visit