bug#53644: 29.0.50; xref-search-program breaks if programm not installed on a remote host

[Philip Kaludercic]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 14.02.2022 15:57, Philip Kaludercic wrote:
>>> And use a simpler test: (only when the host is remote) write some text
>>> to a file in the temp dir, then search through it. Only doing it once,
>>> of course, when the connection-local value is initialized.
>> I am afraid I don't understand what you mean here, specifically "some
>> text".
>> 
>
> Well, we need some file to search and some knowledge about its
> contents in advance (so the search can succeed).

I guess this is what confuses me, what about the contents is relevant to
the query?  `xref-matches-in-files' is already passed a list of files
that can be concatenated into the input for xargs.  The current version
already works and is reasonably fast, so I don't recognise the
improvement.

> Since we don't know anything about the remote system, we probably have
> to create the file ourselves. Put something like "aaa\nbbb\nccc" in
> it, and then search for "bbb".

My apologies, I feel stupid for not understanding, but what would aaa,
bbb and ccc be?

---

Here also the updated version of the patch:

>From 4469098b1dcaefca9b7f190b5e1c1a8ef53791cd Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 9 Feb 2022 10:14:38 +0100
Subject: [PATCH] Check if xref-search-program exists on a remote system

* xref.el (xref--do-search): Loop through all
programmes in xref-search-program-alist as a fallback before raising
an error that the search query couldn't be executed.
(xref-matches-in-files): Extract functionality into xref--do-search
---
 lisp/progmodes/xref.el | 141 ++++++++++++++++++++++++++---------------
 1 file changed, 89 insertions(+), 52 deletions(-)

diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index 6677b4f004..4e4718f5f8 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -1729,6 +1729,79 @@ xref-search-program
   :version "28.1"
   :package-version '(xref . "1.0.4"))
 
+(defun xref--do-search (regexp files dir remotep)
+  "Search for REGEXP in FILES within DIR.
+If REMOTEP is non-nil, gracefully handle the non-existence of the
+preferred searcher (per `xref-search-program'), and attempt to
+restart the query with another program in
+`xref-search-program-alist', only failing if none of these could
+be found either."
+  (pcase-let ((`(,grep-re ,file-group ,line-group . ,_) (car 
grep-regexp-alist))
+              (input (mapconcat #'identity files "\0"))
+              (output (get-buffer-create " *xref grep output*"))
+              (status) (hits))
+    (with-current-buffer output
+      (let* ((default-directory dir)
+             (xref-search-program-alist
+              (if remotep
+                  xref-search-program-alist
+                (list (assoc xref-search-program xref-search-program-alist))))
+             (program (if (bound-and-true-p enable-connection-local-variables)
+                          (with-connection-local-variables
+                           xref-search-program)
+                        xref-search-program))
+             (process-file-side-effects nil))
+        ;; In case `xref-search-program' is not installed on a
+        ;; remote system, we will speculatively try to start a
+        ;; different searcher to see if it is installed and works.
+        (catch 'done
+          (while xref-search-program-alist
+            (erase-buffer)
+            (setq status
+                  (let ((sfn shell-file-name)
+                        (scs shell-command-switch))
+                    (when remotep
+                      (if (bound-and-true-p enable-connection-local-variables)
+                          (with-connection-local-variables
+                           (setq sfn shell-file-name
+                                 scs shell-command-switch))
+                        (setq sfn "/bin/sh")))
+                    (xref--process-file-region
+                     input nil sfn output nil scs
+                     (grep-expand-template
+                      (or (alist-get program xref-search-program-alist)
+                          (user-error "Unknown search program `%s'" program))
+                      (xref--regexp-to-extended regexp)))))
+            (goto-char (point-min))
+            (unless (and (/= (point-min) (point-max))
+                         (not (looking-at grep-re))
+                         ;; TODO: Show these matches as well somehow?
+                         (not (looking-at "Binary file .* matches")))
+              (when (and (not (eq program xref-search-program))
+                         (version<= "27" emacs-version))
+                ;; If possible and necessary, save whatever program
+                ;; was found as a connection local variable.
+                (let* ((host (file-remote-p dir 'host))
+                       (profile (intern (concat "xref-remote-" host))))
+                  (connection-local-set-profile-variables
+                   profile `((xref-search-program . ,program)))
+                  (connection-local-set-profiles
+                   (list :machine host) profile)))
+              (throw 'done t))
+            (setq xref-search-program-alist
+                  (remq (assq program xref-search-program-alist)
+                        xref-search-program-alist)
+                  program (caar xref-search-program-alist)))
+          (user-error "Search failed with status %d: %s" status
+                      (buffer-string)
+                      (buffer-substring (point-min) (line-end-position)))))
+      (while (re-search-forward grep-re nil t)
+        (push (list (string-to-number (match-string line-group))
+                    (match-string file-group)
+                    (buffer-substring-no-properties (point) 
(line-end-position)))
+              hits)))
+    hits))
+
 ;;;###autoload
 (defun xref-matches-in-files (regexp files)
   "Find all matches for REGEXP in FILES.
@@ -1741,69 +1814,33 @@ xref-matches-in-files
   (require 'grep)
   (defvar grep-highlight-matches)
   (pcase-let*
-      ((output (get-buffer-create " *project grep output*"))
-       (`(,grep-re ,file-group ,line-group . ,_) (car grep-regexp-alist))
-       (status nil)
-       (hits nil)
-       ;; Support for remote files.  The assumption is that, if the
+      (;; Support for remote files.  The assumption is that, if the
        ;; first file is remote, they all are, and on the same host.
        (dir (file-name-directory (car files)))
-       (remote-id (file-remote-p dir))
+       (remotep (file-remote-p dir))
        ;; The 'auto' default would be fine too, but ripgrep can't handle
        ;; the options we pass in that case.
-       (grep-highlight-matches nil)
-       (command (grep-expand-template (cdr
-                                       (or
-                                        (assoc
-                                         xref-search-program
-                                         xref-search-program-alist)
-                                        (user-error "Unknown search program 
`%s'"
-                                                    xref-search-program)))
-                                      (xref--regexp-to-extended regexp))))
-    (when remote-id
+       (grep-highlight-matches nil))
+    (when remotep
       (require 'tramp)
       (setq files (mapcar
                    (if (tramp-tramp-file-p dir)
                        #'tramp-file-local-name
-                       #'file-local-name)
+                     #'file-local-name)
                    files)))
     (when (file-name-quoted-p (car files))
       (setq files (mapcar #'file-name-unquote files)))
-    (with-current-buffer output
-      (erase-buffer)
-      (with-temp-buffer
-        (insert (mapconcat #'identity files "\0"))
-        (setq default-directory dir)
-        (setq status
-              (xref--process-file-region (point-min)
-                                         (point-max)
-                                         shell-file-name
-                                         output
-                                         nil
-                                         shell-command-switch
-                                         command)))
-      (goto-char (point-min))
-      (when (and (/= (point-min) (point-max))
-                 (not (looking-at grep-re))
-                 ;; TODO: Show these matches as well somehow?
-                 (not (looking-at "Binary file .* matches")))
-        (user-error "Search failed with status %d: %s" status
-                    (buffer-substring (point-min) (line-end-position))))
-      (while (re-search-forward grep-re nil t)
-        (push (list (string-to-number (match-string line-group))
-                    (match-string file-group)
-                    (buffer-substring-no-properties (point) 
(line-end-position)))
-              hits)))
-    ;; By default, ripgrep's output order is non-deterministic
-    ;; (https://github.com/BurntSushi/ripgrep/issues/152)
-    ;; because it does the search in parallel.
-    ;; Grep's output also comes out in seemingly arbitrary order,
-    ;; though stable one. Let's sort both for better UI.
-    (setq hits
-          (sort (nreverse hits)
-                (lambda (h1 h2)
-                  (string< (cadr h1) (cadr h2)))))
-    (xref--convert-hits hits regexp)))
+    (let ((hits (xref--do-search regexp files dir remotep)))
+      ;; By default, ripgrep's output order is non-deterministic
+      ;; (https://github.com/BurntSushi/ripgrep/issues/152)
+      ;; because it does the search in parallel.
+      ;; Grep's output also comes out in seemingly arbitrary order,
+      ;; though stable one. Let's sort both for better UI.
+      (setq hits
+            (sort (nreverse hits)
+                  (lambda (h1 h2)
+                    (string< (cadr h1) (cadr h2)))))
+      (xref--convert-hits hits regexp))))
 
 (defun xref--process-file-region ( start end program
                                    &optional buffer display
-- 
2.34.0


-- 
        Philip Kaludercic