Re: gnus-search & imap: always "CHARSET UTF-8" when literal+ is supported

[Eric Abrahamsen]

David Edmondson <dme@dme.org> writes:

> Using current emacs git head, talking to outlook.office365.com over
> IMAP.
>
> Attempts to use gnus-search always fail with the server reporting:
>
> (("NO" ("BADCHARSET" "(US-ASCII)") "The" "specified" "charset" "is" "not" 
> "supported."))
>
> Looking at gnus-search.el, `gnus-search-imap-search-command' always
> sends CHARSET UTF-8 if the server supports literal+ (which this one
> does). Sending US-ASCII (or no charset at all) causes the server to
> return the required results in simple test cases.
>
> Is there a way to determine whether a server supports UTF-8 in searches,
> and adjust the command sent accordingly? If not, could the use of UTF-8
> be controlled with (yet another!) variable?

A bit of internet research seems to indicate that Exchange can't handle
UTF-8 encoded search strings, and also there's no way to test that in
advance apart from simply seeing if it errors. Awesome!

I think what this means is that it's impossible to search for non-ascii
text on an Exchange server (can that be true?!). If that's true, then
the imap search command should be using the presence of a multibyte
string as the test for whether to use CHARSET UTF-8 or not. You're not
going to be able to search for a multibyte string, anyway.

Would you try eval'ling the below, and tell me if it works okay when
searching for a string with no non-ascii characters in it?

Also, when you do get the error message above, how does that present to
the user? Did you have to go digging to find it?


(cl-defmethod gnus-search-imap-search-command ((engine gnus-search-imap)
                                               (query string))
  "Create the IMAP search command for QUERY.
Currently takes into account support for the LITERAL+ capability.
Other capabilities could be tested here."
  (with-slots (literal-plus) engine
    (when (and literal-plus
               (string-match-p "\n" query))
      (setq query (split-string query "\n")))
    (cond
     ((consp query)
      ;; We're not really streaming, just need to prevent
      ;; `nnimap-send-command' from waiting for a response.
      (let* ((nnimap-streaming t)
             (call
              (nnimap-send-command
               "UID SEARCH CHARSET UTF-8 %s"
               (pop query))))
        (dolist (l query)
          (process-send-string (get-buffer-process (current-buffer)) l)
          (process-send-string (get-buffer-process (current-buffer))
                               (if (nnimap-newlinep nnimap-object)
                                   "\n"
                                 "\r\n")))
        (nnimap-get-response call)))
     (t (nnimap-command "UID SEARCH %s" query)))))