Re: Refrain from immediate `delete-process` in `tramp-cleanup-all-connections`

[Jordan Ellis Coppard]

Hello Michael another long one from me,


I realise backtick/grave ` has meaning in elisp but habit (and I can't 
think of a good alternative) makes me use it to delimit code, so unless 
specified no quoted forms are intended when I use ` here hehe!

So for the customisation things, I looked at the documentation again and 
I got something basic scaffolded in principle but ran into a tangential 
issue while doing so, I'll explain for full context (and I have the code 
afterwards):

I want that custom filter command as mentioned, then I also thought 
about altering how the names of the containers appear in the completion 
list and during that process I couldn't figure out how to nicely NOT use 
prior connection history and other automatically parsed connection 
information. While testing this I noticed after I remove my Tramp 
history file manually and restart Emacs that some bogus Tramp cache data 
was still present.



(1) `save-place-mode` clobbers Tramps cache

It turns out that `save-place-mode` is corrupting Tramp's cache. My 
guess is that when Emacs starts and `save-hist-mode` loads its history 
file which is called `places` (by default in the users Emacs 
configuration directory) that the Tramp file name handler is triggered 
as said `places` file is a literal elisp file with a list of cons pairs 
of file name and data (point, dired info etc), for example:


;;; -*- coding: utf-8; mode: lisp-data -*-
(("/podmancp:quizzical_tharp:/home/jammy/project/build.zig" . 1474))


Loading Emacs with the Tramp cache file deleted, and with an Emacs 
configuration that enables save-hist-mode with `places` as above and 
then typing `C-x C-f /podmancp:` lists a real container `jam-zevem`, and 
`quizzical_tharp` the latter of which long-since hasn't existed.

If I do the same but delete my `places` and `tramp` history/cache files 
only `jam-zevem` is listed, and when Emacs is closed the resulting Tramp 
cache is minimal with the expected values. If Tramp has populated its 
cache from `places` the saved Tramp cache `tramp` is polluted with 
entries which could make zero sense with no easy/obvious way for the 
user to remove them since they'll just be loaded again implicitly via 
`save-hist-mode`.

I suspect this might also cause hangs in the wild for users who use 
`save-hist-mode` too. It's not enabled by default but as I'm sure you've 
guessed is in my config.

Disabling save-hist-mode integration (or changing it so it doesn't 
clobber Tramp) by default is the solution here I think; since 
save-hist-mode looks like it can (by default) keep history that is very 
ancient. Optional integration allowing users to opt-in would prevent 
nasty surprises like my case here.



(2) Tramp and forgetting projects

If I attempt to forget a project.el project with 
`project-forget-project` Tramp will attempt to establish a connection 
which can be frustrating if the project doesn't exist anymore (moved 
directories, changed host configuration, any number of reasons) and not 
only causes a pause in the best-case (during interactive minibuffer 
completion) but in worst case can freeze Emacs until C-g is issued. I 
believe a timeout via `remote-file-name-access-timeout` can be 
configured but a user would still then have to wait the timeout time (or 
know to press C-g) just to forget a project. I might be ignorant of 
potential hooks people may(not) use in which case Tramp probably should 
connect so those hooks can run (imagine a hook to delete the files 
executes when a project is forgotten). Perhaps (if those sorts of things 
do exist/occur) Tramp should only attempt to connect after interactive 
completion has returned a value, and not simply while browsing with C-n 
and C-p; or have it configurable.

I re-define `project-forget-project` in my init.el which is the same as 
the default implementation but with `non-essential` lexically bound to 
stop Tramp doing this:


(defun project-forget-project (project-root)
  "Remove directory PROJECT-ROOT from the project list.
PROJECT-ROOT is the root directory of a known project listed in
the project list."
  (interactive (list (funcall project-prompter)))
  (let ((non-essential t)) ; JORDAN: My single change, rest is verbatim 
from project.el sources.
        (project--remove-from-project-list
         project-root "Project `%s' removed from known projects")))


I see other areas of project.el use non-essential in consideration of 
possible Tramp interaction, perhaps Tramp and/or project.el need 
adjustments here too (or maybe my single line patch here is such a 
solution for upstream; albeit defaults to always never connecting).


(3) Blessed custom approach clobbers the method it uses

Now I know why certain completions are appearing in my minibuffer which 
shouldn't be (save-hist-mode clobbering Tramp cache) I got back to the 
customised podmancp connection stuff.

While "(tramp) Customizing Completion" does outline a way it only seems 
possible to have the customised completion take effect all the time, or 
none of the time (will get back to this).

These development "jam" containers have a well-known user `jammy`, and 
to avoid having to type /podmancp:jammy@ all the time I made an 
abbreviation as Tramp documents:


(defun tjp/abbrev-no-expand-char () t)
(put 'tjp/abbrev-no-expand-char 'no-self-insert t)

(define-abbrev-table 'my-tramp-abbrev-table
  '(("jam" "/podmancp:jammy@" tjp/abbrev-no-expand-char)))

(add-hook
 'minibuffer-setup-hook
 (lambda ()
   (abbrev-mode 1)
   (setq local-abbrev-table my-tramp-abbrev-table)))

(advice-add 'minibuffer-complete
                        :before 'expand-abbrev)


This is fine and it does work -- I note that the Tramp docs may also 
want to include the `'no-self-insert` snippet otherwise after expanding 
the abbreviation one must backspace before being able to type (or view) 
completions.

The problem arises whereby I have no nice way (that I see/understand 
currently) to conditionally apply my custom filter. The filters apply to 
the podmancp method which includes podmancp:jammy or 
podmancp:foo:~/foo/bar or podmancp:lorem@ipsum:/root/foo.txt and so 
forth; so long as the method is podmancp either I must always use my 
custom filter (bad, now I've clobbered podmancp globally) or I don't 
have access to it.

Said custom filter (the single change is in the arguments to 
shell-command-to-string) is:


(defun tjp/jam--tramp-completion-function (method)
  (tramp-skeleton-completion-function method
        (when-let* ((raw-list
                                 (shell-command-to-string
				  (concat program " ps -a --filter 'label=sh.jammy.box' --format 
'{{.ID}}\t{{.Names}}'")))
                                (lines (split-string raw-list "\n" 'omit))
                                (names
                                 (tramp-compat-seq-keep
                                  (lambda (line)
                                        (when (string-match
                                                   (rx bol (group (1+ nonl))
							   ;; Could remove prefix if Tramp connection login args add it 
back e.g. jam-%h. Maybe Emacs completion backend has annotated values 
which could do this also but the jam prefix is intended to be a very 
"hardcoded" value and not fluid. Jam containers are dev environments.
                                                           ;; "\t" (? "jam-") 
(? (group (1+ nonl))) eol)
                                                           "\t" (? (group (1+ 
nonl))) eol)
                                                   line)
                                          (or (match-string 2 line) 
(match-string 1 line))))
                                  lines)))
          (mapcar (lambda (name) (list nil name)) names))))

(tramp-set-completion-function "podmancp" 
'((tjp/jam--tramp-completion-function "podmancp")))


So if there were a way to use this completion function for a specific 
method/host/user combination, or alternately regexp similarly to how one 
can define different connection properties as "(tramp) Predefined 
connection information") explains I think this would be solved.

While that is also a lot of code and requires consulting abbrev-mode the 
alternative which works today and doesn't clobber podmancp is creating a 
new method entirely (backtick ` is for elisp in this snippet):


(setq jam--devbox
          `("jam"
                (tramp-login-program ,tramp-podman-method)
        (tramp-login-args (("exec")
                           ("-it")
                           ("-u" "jammy")
                                                   ("--workdir" 
"/home/jammy/project")
                           ("%h")
                                                   ("%l")))
                (tramp-direct-async (,tramp-default-remote-shell "-c"))
        (tramp-remote-shell ,tramp-default-remote-shell)
        (tramp-remote-shell-login ("-l"))
        (tramp-remote-shell-args ("-i" "-c"))
                (tramp-copy-program ,tramp-podman-method)
                (tramp-copy-args (("cp")))
                (tramp-copy-file-name (("%h" ":") ("%f")))
        (tramp-copy-recursive t)))

(tramp-set-completion-function "jam" 
`((tjp/jam--tramp-completion-function "jam")))

(add-to-list 'tramp-methods jam--devbox)


The disadvantage here being that one must either know all of what to 
define for new Tramp methods or learn that (by looking at Tramp's source 
code as I did) and still most is a mere copy-paste. Also completion 
feels better since typing /jam: doesn't require managing abbreviations, 
manually expanding them, and adding logic to remove the expansion character.

Finally while trying to conditionally apply things, and while I was 
trying to figure out where the clobbered completions (now known from 
save-hist-mode) were coming from I was doing things like this:


(defconst bingbong ((tramp-set-completion-function "podmancp" 
'((tjp/jam--tramp-completion-function "podmancp")))))
;; (defconst bingbong (list '(tramp-completion-use-auth-sources nil) 
'(tramp-completion-use-cache t)))
;; (defconst bingbong (list '(tramp-completion-use-auth-sources t) 
'(tramp-completion-use-cache nil)))

;; (setq tramp-completion-use-cache t)
;; (setq tramp-completion-use-auth-sources t)

(connection-local-set-profile-variables
 'bingbong-prof
 bingbong)

(connection-local-set-profiles
 `(:application tramp :protocol "podmancp" :user "jammy")
 'bingbong-prof)


If that is the blessed route this becomes quite a lot of code surface 
area for something that feels like it should be simple, and in any case 
the interplay between when connection local variables are applied and 
when Tramp reads them, Tramps own connection properties and when they 
are available/read etc is a bit complex to me still.

I had success with the immediate above snippet.. I think.. because I 
could never get it consistent and using M-x profiler to help determine 
backtraces etc, or short of stepping through with edebug et al; or 
reading much more documentation versus just using a complete custom 
method and saving a lot of time. I am for reading documentation of 
course, but getting into the weeds of connection-local variables and 
friends (in addition to abbrev-mode before) is another fork in the path 
which builds up. In this specific case it's quite trivial too: just 
being able to list containers with a specific label.

That said having to make a custom method every time isn't very 
composable, perhaps doing all of this "at Tramp" is the wrong area and 
some external successive filtering is required. Maybe some kind of 
Orderless (or alternative) completion style so that if I type (for 
example) ?jam Ordlerless further filters.. I don't know. The domain of 
responsibility here is murky to me.



(4) delete-process

I still think delete-process (original topic in this thread) should be 
changed hehe; my fault for adding more topics continually and in this 
very reply too!


On 19/1/2025 8:26 pm, Michael Albinus wrote:
> Well. For more than 20 years, I'm the Tramp maintainer. Other people do
> contribute (I'm very grateful for this!), but in general this is a
> one-man-show. I would appreciate much, if somebody (you?) could
> contribute on a regular basis.

If I have things to contribute I'd love to; I'm already behind on a 
bunch of other work currently and I don't feel like I understand more 
than the trivial parts of Tramp but if there's something in future that 
I can I will try.

I will say when I first checked the tramp-devel mailing list the low 
activity (for whatever reason, and its my internalisation) made me think 
about trying harder to solve issues before sending an email only because 
I imagine (and you confirm) Tramp is probably a one-man-show and I think 
I would've felt bad bothering so eagerly.. but then I tried too hard and 
frustrated myself. You can imagine the extreme end of eagerness is to 
immediately send a "how do I use this" to the mailing list (instead of 
reading the documentation) -- it's tough to balance trying to hack a 
solution together yourself versus when it's time to just email asking 
for help. I expect your opinion is "just ask if you're stuck!".

> PS: Tramp, like Emacs, is copyrighted to the FSF. If somebody wants to
> contribute more than a small change (up to 15 lines), we need to sign
> legal papers for the FSF copyrights. Are you willing to do so?

If you want to do this ahead of time I am willing, or at the time of 
first contribution (if any) it makes no difference to me.


/Jordan