Re: Faster start-file-process?

[Philipp Stephani]

Thanks for the quick response!

Am Di., 10. März 2020 um 10:09 Uhr schrieb Michael Albinus
<address@hidden>:
>
> Philipp Stephani <address@hidden> writes:
>
> > Hi,
>
> Hi Philippe,
>
> > I managed to speed up start-file-process for SSH by a factor of 500
> > (on my machine) by enabling SSH control mode in ~/.ssh/config and then
> > just calling
> > (start-process name buffer "ssh" program args).
> > Such a speedup is crucial for using start-file-process for truly
> > asynchronous operations that may not block, e.g., Flymake backends.
> > Are there any downsides to my approach? Would if be possible to
> > include it in Tramp itself?
>
> Per default, Tramp uses ssh ControlMaster if possible. What are the
> values of `tramp-use-ssh-controlmaster-options' and
> `tramp-ssh-controlmaster-options' in your case, after connecting to a
> remote host?

t and "-o ControlMaster=auto -o ControlPath='tramp.%%C' -o
ControlPersist=no" (i.e. the defaults).

>
> Could you set `tramp-verbose' to 6 and verify, whether these options are
> used? You could send the traces to me for analyzing, if you prefer.

They seem to be used, the output is:

Tramp: Opening connection for HOSTNAME using ssh...
Tramp: Sending command ‘exec ssh -q   -o ControlMaster=auto -o
ControlPath='tramp.%C' -o ControlPersist=no -e none HOSTNAME’
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on ‘HOSTNAME’
Tramp: Opening connection for HOSTNAME using ssh...done

The form I've evaluated is just
(let ((default-directory "/ssh:HOSTNAME:/"))
    (start-file-process "" nil "true"))

This takes 2 seconds per invocation on my machine:
(benchmark-run 10
  (let ((default-directory "/ssh:HOSTNAME:/"))
    (start-file-process "" nil "true")))
(21.287248981 78 6.104474747000001)

If you're interested, I've profiled Tramp with ELP (on Emacs master),
here's the report:

tramp-file-name-handler                            79
48.196645538  0.6100841207
tramp-sh-file-name-handler                         59
47.933230473  0.8124276351
tramp-handle-start-file-process                    11
23.966547321  2.1787770292
tramp-sh-handle-make-process                       11
23.916336605  2.1742124187
tramp-maybe-open-connection                        77
23.538912846  0.3057001668
tramp-process-actions                              11
16.095361023  1.4632146384
tramp-check-for-regexp                             27300
13.363068022  0.0004894896
tramp-search-regexp                                27520
8.4726394889  0.0003078720
tramp-send-command                                 198
7.0239918969  0.0354747065
tramp-wait-for-output                              176
6.898226061   0.0391944662
tramp-accept-process-output                        2955
6.8919202430  0.0023322911
tramp-wait-for-regexp                              187
6.8587603229  0.0366778626
tramp-open-connection-setup-interactive-shell      11
6.817871086   0.6198064623
tramp-get-connection-property                      56391
2.8474465550  5.049...e-05
tramp-send-command-and-check                       44
1.8092037689  0.0411182674
tramp-barf-unless-okay                             33
1.345802245   0.0407818862
tramp-send-command-and-read                        33
1.3418833959  0.0406631332
tramp-debug-message                                785
1.1342730509  0.0014449338
tramp-open-shell                                   11
1.106302197   0.1005729269
tramp-get-hash-table                               56732
0.7261888440  1.280...e-05
[...]

I'm not sure what the culprit is, but my method (that doesn't use
Tramp's connections) is reliably 500 times faster.