Re: Improve performances on high latency connexion

[Compostella, Jeremy]

Michael Albinus <address@hidden> writes:

> address@hidden (Compostella, Jeremy) writes:
>
>> Hi,
>
> Hi Jérémy,
>
>> Running a grep over tramp command takes more than 30 seconds so I
>> investigated a little bit today.  I found out that file properties has a
>> cache that I could use by customizing the
>> `remote-file-name-inhibit-cache' variable.  Its makes me save a bit more
>> than 10 seconds (according to ELP).
>
> Yep. However, you are responsible then that no remote file you are
> working on is changed outside Emacs.
That's true, but when I'm working from it's worth it.

>> I investigated a little bit more and figured out that a lot of shell
>> commands are sent especially to set environment variables.  I've
>> gathered these set variable commands in two commands and according to
>> ELP I've saved another 12 seconds.  Now running grep from Emacs over
>> tramp gets me result in less than 10 seconds which become acceptable.
>>
>> I've joined the patch I've made, could you please review it (merge it
>> ?).
>
> Thanks for your patch, it makes sense. Just one comment:
>
>> @@ -4078,15 +4076,17 @@ process to set up.  VEC specifies the connection."
>>    (let ((env (append (when (tramp-get-remote-locale vec) ; Discard `(nil)'.
>>                     `(,(tramp-get-remote-locale vec)))
>>                   (copy-sequence tramp-remote-process-environment)))
>> -    unset item)
>> +    unset vars item)
>>      (while env
>>        (setq item (tramp-compat-split-string (car env) "="))
>>        (setcdr item (mapconcat 'identity (cdr item) "="))
>>        (if (and (stringp (cdr item)) (not (string-equal (cdr item) "")))
>> -      (tramp-send-command
>> -       vec (format "%s=%s; export %s" (car item) (cdr item) (car item)) t)
>> +      (push (format "%s=%s; export %s" (car item) (cdr item) (car item)) 
>> vars)
>>      (push (car item) unset))
>>        (setq env (cdr env)))
>> +    (when vars
>> +      (tramp-send-command
>> +       vec (mapconcat 'identity vars " ; ")))
>>      (when unset
>>        (tramp-send-command
>>         vec (format "unset %s" (mapconcat 'identity unset " ")) t))))
>
> This sends just one line to the remote shell. We don't know how long,
> and people could expand the number of environment variables to be set.
>
> Sometimes, shells are restricted in the length of a line they
> accept. Therefore, it might be better to wrap these settings in a
> heredoc command, which does not suffer from this limitation.
I figured this out yersterday but couldn't find time to work on it.
Thanks.

> Does the following changed patch work for you? Same optimization result?
>
> *** /home/albinus/src/tramp/lisp/tramp-sh.el.~master~ 2014-10-13 
> 21:51:07.480159402 +0200
> --- /home/albinus/src/tramp/lisp/tramp-sh.el  2014-10-13 21:06:30.285337686 
> +0200
> ***************
> *** 3955,3964 ****
>   
>     (tramp-message vec 5 "Setting shell prompt")
>     (tramp-send-command
> !    vec (format "PS1=%s" (tramp-shell-quote-argument tramp-end-of-output)) t)
> !   (tramp-send-command vec "PS2=''" t)
> !   (tramp-send-command vec "PS3=''" t)
> !   (tramp-send-command vec "PROMPT_COMMAND=''" t)
>   
>     ;; Try to set up the coding system correctly.
>     ;; CCC this can't be the right way to do it.  Hm.
> --- 3955,3962 ----
>   
>     (tramp-message vec 5 "Setting shell prompt")
>     (tramp-send-command
> !    vec (format "PS1=%s PS2='' PS3='' PROMPT_COMMAND=''"
> !            (tramp-shell-quote-argument tramp-end-of-output)) t)
>   
>     ;; Try to set up the coding system correctly.
>     ;; CCC this can't be the right way to do it.  Hm.
> ***************
> *** 4078,4092 ****
>     (let ((env (append (when (tramp-get-remote-locale vec) ; Discard `(nil)'.
>                      `(,(tramp-get-remote-locale vec)))
>                    (copy-sequence tramp-remote-process-environment)))
> !     unset item)
>       (while env
>         (setq item (tramp-compat-split-string (car env) "="))
>         (setcdr item (mapconcat 'identity (cdr item) "="))
>         (if (and (stringp (cdr item)) (not (string-equal (cdr item) "")))
> !       (tramp-send-command
> !        vec (format "%s=%s; export %s" (car item) (cdr item) (car item)) t)
>       (push (car item) unset))
>         (setq env (cdr env)))
>       (when unset
>         (tramp-send-command
>          vec (format "unset %s" (mapconcat 'identity unset " ")) t))))
> --- 4076,4097 ----
>     (let ((env (append (when (tramp-get-remote-locale vec) ; Discard `(nil)'.
>                      `(,(tramp-get-remote-locale vec)))
>                    (copy-sequence tramp-remote-process-environment)))
> !     unset vars item)
>       (while env
>         (setq item (tramp-compat-split-string (car env) "="))
>         (setcdr item (mapconcat 'identity (cdr item) "="))
>         (if (and (stringp (cdr item)) (not (string-equal (cdr item) "")))
> !       (push (format "%s %s" (car item) (cdr item)) vars)
>       (push (car item) unset))
>         (setq env (cdr env)))
> +     (when vars
> +       (tramp-send-command
> +        vec
> +        (format "while read var val; do export $var=$val; done 
> <<'%s'\n%s\n%s"
> +            tramp-end-of-heredoc
> +            (mapconcat 'identity vars "\n")
> +            tramp-end-of-heredoc)
> +        t))
>       (when unset
>         (tramp-send-command
>          vec (format "unset %s" (mapconcat 'identity unset " ")) t))))
>
Looks good to me :)

>> I have a question too: is there a way to use a "persistent" tramp
>> connexion to run commands ?
>
> Emacs knows synchronous and asynchronous processes. A "persistent tramp
> connection" would be implemented by a synchronous process.
>
> However, Emacs' grep command uses an asynchronous process.
OK, thanks.


>> Thanks,
>>
>> Jérémy
>
> Best regards, Michael.
>

-- 
One Emacs to rule them all
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.