bug#49264: 28.0.50; project.el+tramp performance issue

[Ergus]

On Mon, Aug 09, 2021 at 03:59:55AM +0300, Dmitry Gutov wrote:
> Hi again,
>
> On 27.07.2021 02:04, Dmitry Gutov wrote:
>
> Sorry for the long wait, this feature ties into another old 
> discussion, and I wasn't sure how to proceed best.
>
> Anyway, here's a patch to try (attached). It should recover 
> performance lost back in 4ca13d98c9e while retaining flexibility (and 
> even adding more).
>
> There are still less than optimal places there (e.g. 
> file-in-directory-p is not very fast), and the modules list is read 
> from the disk every time, so some further optimization should be 
> possible.
>
> But first please try this version.

Thanks Dmitry:

This patch reduced the C-x p b time to just a few (~5) seconds when I
have like 20 remote buffers.

At the moment I haven't have time to stress it a bit more, but it
improves the situation significantly compared to the previous situation.

In my opinion this is a good improvement and may be installed on master,
but probably it is not enough.

I made a manual fast profiling and I see that most of the time in
project-buffers actually goes to tramp-sh-file-name-handler.

         207  42%         - project-buffers
         207  42%          - apply
         207  42%           - #<compiled -0x1f5d919efefc0a09>
         200  40%            - file-in-directory-p
         200  40%             - tramp-file-name-handler
         197  40%              - apply
         197  40%               - tramp-sh-file-name-handler
         197  40%                - tramp-handle-file-in-directory-p
         183  37%                 - tramp-run-real-handler
         183  37%                  - file-in-directory-p
         124  25%                   - file-equal-p
         124  25%                    - tramp-file-name-handler
         121  24%                     - apply
         121  24%                      - tramp-sh-file-name-handler
         121  24%                       - tramp-handle-file-equal-p
          85  17%                        - tramp-run-real-handler
          85  17%                         - file-equal-p
          52  10%                          - file-truename
          52  10%                           - tramp-file-name-handler
          41   8%                            - apply
          41   8%                             - tramp-sh-file-name-handler
          41   8%                              - tramp-sh-handle-file-truename
          28   5%                               + file-remote-p
          10   2%                               + file-local-name
           3   0%                               + file-name-as-directory

It goes specifically to file-in-directory-p as you said. So maybe the
improvement there may be also desirable if the difference after the
optimization can reduce the time for file-in-directory-p (or the caller)
at least to the half.

So, very thanks again.

Ergus.