Re: Improve performances on high latency connexion

[Michael Albinus]

address@hidden (Compostella, Jeremy) writes:

> Hi,

Hi Jérémy,

> Running a grep over tramp command takes more than 30 seconds so I
> investigated a little bit today.  I found out that file properties has a
> cache that I could use by customizing the
> `remote-file-name-inhibit-cache' variable.  Its makes me save a bit more
> than 10 seconds (according to ELP).

Yep. However, you are responsible then that no remote file you are
working on is changed outside Emacs.

> I investigated a little bit more and figured out that a lot of shell
> commands are sent especially to set environment variables.  I've
> gathered these set variable commands in two commands and according to
> ELP I've saved another 12 seconds.  Now running grep from Emacs over
> tramp gets me result in less than 10 seconds which become acceptable.
>
> I've joined the patch I've made, could you please review it (merge it
> ?).

Thanks for your patch, it makes sense. Just one comment:

> @@ -4078,15 +4076,17 @@ process to set up.  VEC specifies the connection."
>    (let ((env (append (when (tramp-get-remote-locale vec) ; Discard `(nil)'.
>                      `(,(tramp-get-remote-locale vec)))
>                    (copy-sequence tramp-remote-process-environment)))
> -     unset item)
> +     unset vars item)
>      (while env
>        (setq item (tramp-compat-split-string (car env) "="))
>        (setcdr item (mapconcat 'identity (cdr item) "="))
>        (if (and (stringp (cdr item)) (not (string-equal (cdr item) "")))
> -       (tramp-send-command
> -        vec (format "%s=%s; export %s" (car item) (cdr item) (car item)) t)
> +       (push (format "%s=%s; export %s" (car item) (cdr item) (car item)) 
> vars)
>       (push (car item) unset))
>        (setq env (cdr env)))
> +    (when vars
> +      (tramp-send-command
> +       vec (mapconcat 'identity vars " ; ")))
>      (when unset
>        (tramp-send-command
>         vec (format "unset %s" (mapconcat 'identity unset " ")) t))))

This sends just one line to the remote shell. We don't know how long,
and people could expand the number of environment variables to be set.

Sometimes, shells are restricted in the length of a line they
accept. Therefore, it might be better to wrap these settings in a
heredoc command, which does not suffer from this limitation.

Does the following changed patch work for you? Same optimization result?

--8<---------------cut here---------------start------------->8---
*** /home/albinus/src/tramp/lisp/tramp-sh.el.~master~   2014-10-13 
21:51:07.480159402 +0200
--- /home/albinus/src/tramp/lisp/tramp-sh.el    2014-10-13 21:06:30.285337686 
+0200
***************
*** 3955,3964 ****
  
    (tramp-message vec 5 "Setting shell prompt")
    (tramp-send-command
!    vec (format "PS1=%s" (tramp-shell-quote-argument tramp-end-of-output)) t)
!   (tramp-send-command vec "PS2=''" t)
!   (tramp-send-command vec "PS3=''" t)
!   (tramp-send-command vec "PROMPT_COMMAND=''" t)
  
    ;; Try to set up the coding system correctly.
    ;; CCC this can't be the right way to do it.  Hm.
--- 3955,3962 ----
  
    (tramp-message vec 5 "Setting shell prompt")
    (tramp-send-command
!    vec (format "PS1=%s PS2='' PS3='' PROMPT_COMMAND=''"
!              (tramp-shell-quote-argument tramp-end-of-output)) t)
  
    ;; Try to set up the coding system correctly.
    ;; CCC this can't be the right way to do it.  Hm.
***************
*** 4078,4092 ****
    (let ((env (append (when (tramp-get-remote-locale vec) ; Discard `(nil)'.
                       `(,(tramp-get-remote-locale vec)))
                     (copy-sequence tramp-remote-process-environment)))
!       unset item)
      (while env
        (setq item (tramp-compat-split-string (car env) "="))
        (setcdr item (mapconcat 'identity (cdr item) "="))
        (if (and (stringp (cdr item)) (not (string-equal (cdr item) "")))
!         (tramp-send-command
!          vec (format "%s=%s; export %s" (car item) (cdr item) (car item)) t)
        (push (car item) unset))
        (setq env (cdr env)))
      (when unset
        (tramp-send-command
         vec (format "unset %s" (mapconcat 'identity unset " ")) t))))
--- 4076,4097 ----
    (let ((env (append (when (tramp-get-remote-locale vec) ; Discard `(nil)'.
                       `(,(tramp-get-remote-locale vec)))
                     (copy-sequence tramp-remote-process-environment)))
!       unset vars item)
      (while env
        (setq item (tramp-compat-split-string (car env) "="))
        (setcdr item (mapconcat 'identity (cdr item) "="))
        (if (and (stringp (cdr item)) (not (string-equal (cdr item) "")))
!         (push (format "%s %s" (car item) (cdr item)) vars)
        (push (car item) unset))
        (setq env (cdr env)))
+     (when vars
+       (tramp-send-command
+        vec
+        (format "while read var val; do export $var=$val; done <<'%s'\n%s\n%s"
+              tramp-end-of-heredoc
+              (mapconcat 'identity vars "\n")
+              tramp-end-of-heredoc)
+        t))
      (when unset
        (tramp-send-command
         vec (format "unset %s" (mapconcat 'identity unset " ")) t))))
--8<---------------cut here---------------end--------------->8---

> I have a question too: is there a way to use a "persistent" tramp
> connexion to run commands ?

Emacs knows synchronous and asynchronous processes. A "persistent tramp
connection" would be implemented by a synchronous process.

However, Emacs' grep command uses an asynchronous process.

> Thanks,
>
> Jérémy

Best regards, Michael.