bug#24478: 25.1; Regression in 25.1: .tramp_history files are littered in non-$HOME working directories

[Ævar Arnfjörð Bjarmason]

On Tue, Oct 11, 2016 at 3:54 PM, Michael Albinus <michael.albinus@gmx.de> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
> Hi Ævar,
>
>> I'm the reporter, so I obviously have a dog in this fight, but I don't
>> think that makes sense. This whole facility introduced in the emacs-25
>> series still seems really broken since its introduction, and the
>> various regressions reported have just resulted in other regressions
>> taking their place, the latest one being discussed in this ticket.
>
> I'm also unhappy about this story. I really would like to use a proper
> and robust default value for this. But there isn't one so far.
>
>>  * In emacs-24 there was no way to have a Tramp history file, we'd
>> just specify a HISTFILE=/dev/null environment variable.
>
> This was introduced back in 2014. Before this change, HISFILE was unset
> somewhere else in the initialization hand-shake, but at a later
> point. It didn't work properly then.
>
>>  * 9be1538 added an option to change that, so you could have a history
>> file as a file, defaulting to /dev/null, but they way it was
>> implemented caused it to unlink /dev/null, as reported in
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=19731
>
> Bug#19371 has reported, that there is a bash bug
> <https://bbs.archlinux.org/viewtopic.php?pid=1397412#p1397412> which has
> this effect. It is no Tramp error, and I would regard this setting still
> be the best one if possible. But due to this bash bug, this setting
> would damage the remote system. So we cannot use "/dev/null" as default,
> even if the bash bug has been fixed. There will still be system in the
> wild with this bug.
>
>>  * So Michael patched it to make 'unset an option, which was
>> implemented in 6f8372d, as far as I can tell at this point the
>> facility worked the way it did in emacs-24 again. I.e. no history by
>> default, but no regression with unlinking /dev/null
>>
>>  * 'unset was made the default by Michael in 954ca0f, but just a few
>> hours later this was set to t instead in c10828b, which does the same
>> thing as 'unset according to the commit message. I.e. just an internal
>> refactoring. This was followed-up by 24fa4ff to refactor it some more.
>
> Yes.
>
>> * It was then changed from t to ".tramp_history" in 1e04ea9. The
>> commit message says to fix
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20446 but I don't see
>> how it could eat the bash history if it's set to not have any history
>> file by default.
>
> Glenn did report, that in his use case unsetting HISTFILE has changed
> his ~/.bash_history to a zero size. Not acceptable, and again a special
> behaviour of bash :-(
>
> So the only solution I could thing about is setting this variable to a
> Tramp specific value.
>
>> * Now because it's ".tramp_history" and not "~/.tramp_history" it gets
>> created in random non-~ directories you open with tramp, but more
>> importantly, and I didn't realize this in my initial report, the shell
>> history *might be shared between multiple users*, which seems like a
>> bad security issue.
>
> "~/.tramp_history" would be the obvious choice, but "~/" is not
> guaranteed to exist. An example is hydra, were the tests failed with
> this setting.
>
>> It seems to me that the best solution to this whole problem is to set
>> it to "t" again which would return to the non-history days of
>> emacs-24, since apparently using ~ can't be counted on.
>
> How do you want explain it to bash users like Glenn? Their history file
> will get lost, again.

Thanks for all the follow-up details. I mainly included all of that to
get all this straight in my head, but I was apparently wrong about it
being viable to revert to the old behavior vis-a-vis unlinking
/dev/null on those old systems.

>> In addition, depending on the bug with history potentially being
>> shared between users now that it's being dumped in random potentially
>> shared FS directories they open with tramp, changing this to
>> ".tramp_history" might have caused a security issue worth of a CVE,
>> but I haven't investigated that, but we *certainly* went from no
>> history by default in emacs-24 to history littered in potentially
>> world readable directories in emacs-25.
>
> I still don't understand why the ".tramp_history" file is spread over
> the file system. This setting is apllied immediately after connecting to
> the remote host. I would assume that one lands in the home directory
> there; ".tramp_history" should be expanded relatively to that directory.
>
> Could you show hot it happens to you that it is expanded to another
> place? Pls run Tramp from scratch, after increasing the debug level by
>
> (setq tramp-verbose 6)
>
> There will be a Tramp debug buffer, which might tell us what happens.
>
> Best regards, Michael.

All I'm doing is e.g.:

1. C-x C-f //ssh:puppet-staging:/etc/puppet/environment/aearnfjord/puppet RET
2. Run some command e.g. M-x grep <hi>
3. This results in:
    $ wc -l $PWD/.tramp_history
    39 /etc/puppet/environments/aearnfjord/puppet/.tramp_history

I've snipped away the parts I think are irrelevant because it's very
verbose, but this hopefully the relevant tramp-verbose=6 output:

[I don't have an existing tramp connection at this point, so it connects]
16:24:08.497153 tramp-process-one-action (5) # Looking for regexp "\(^[^#$%>
]*[#$%>] *\)\'" from remote shell
16:24:08.497248 tramp-process-one-action (5) # Call ‘tramp-action-succeed’
16:24:08.497369 tramp-process-actions (6) #
nohup: redirecting stderr to stdout
Last login: Tue Oct 11 16:03:15 2016 from <host>^M
[...]
^[[1;31mPROD^[[m puppet-staging ~ (DUMMY) $
16:24:08.497762 tramp-process-actions (3) # Waiting for prompts from
remote shell...done
16:24:08.498117 tramp-maybe-open-connection (3) # Found remote shell
prompt on ‘puppet-staging’
16:24:08.498251 tramp-open-shell (5) # Opening remote shell ‘/bin/sh’...
16:24:08.498387 tramp-send-command (6) # exec env ENV=''
HISTFILE=.tramp_history PROMPT_COMMAND='' PS1=\#\$\  PS2='' PS3=''
/bin/sh
16:24:08.522855 tramp-wait-for-regexp (6) #
#$
16:24:08.522995 tramp-open-shell (5) # Opening remote shell ‘/bin/sh’...done
16:24:08.523075 tramp-open-connection-setup-interactive-shell (5) #
Setting up remote shell environment
16:24:08.523157 tramp-send-command (6) # stty tab0 -inlcr -onlcr -echo
kill '^U' erase '^H'
16:24:08.543347 tramp-wait-for-regexp (6) #
[...]

Then this and all subsequent commands end up in my non-~ .tramp_history:

$ head -n 2 /etc/puppet/environments/aearnfjord/puppet/.tramp_history
#1476195869
stty tab0 -inlcr -onlcr -echo kill '^U' erase '^H'

If this isn't the relevant output or you'd like to me to try something
else I'd be happy to provide more details.