Re: Multi-tty design (Re: Reordering etc/NEWS)

[David Kastrup]

Karoly Lorentey <address@hidden> writes:

> David Kastrup wrote:
>> Karoly Lorentey <address@hidden> writes:
>>> No, my memory has failed me.  I had a patch implementing the above
>>> design, but what we currently have in the tree is something more
>>> complex: environment variables are neither frame-, nor
>>> terminal-local, but rather client-local.
>> 
>> I have seen on the archives of the multi-tty list and its README that
>> the implementation has went through several different ideas.  So quite
>> a bit of work has been invested already, and there is obviously not
>> going to be much enthusiasm for scrapping it.
>
> Sure, but better scrap parts than drop the whole thing.  As you say,
> environments have already gone through several reimplementations.  I
> certainly don't mind having another iteration. :-)  What we have now is
> my personal favourite, but that doesn't mean it's the best for everyone.
>
> I argue that while it is a valid viewpoint to say that things such as
> CFLAGS or TEXINPUTS should always come from the original environment, it
> is equally valid and defensible to say that they should come from the
> local terminal.

But things like a shell-buffer are not tied to a terminal (and most
certainly not to a frame).  Neither are process-buffers.  And yet you
need to start processes in them, and they need to have an environment
which you can more or less reliably _set_.

> (Emacs may have been running in the background for weeks, and I may
> have just started working on my brand new TeX file in a recently
> started emacsclient session.)  Both viewpoints should be catered
> for.

I disagree.  If a viewpoint can't be catered for without breaking a
_lot_ of things and guarantees, catering for it might be a bad idea.

> Fair enough.  So let's do terminal-local environments.  I hereby
> withdraw the current environment implementation, and propose the
> following simple solution instead:
>
>       - Make `process-environment' a terminal-local variable
>         (with DEFVAR_KBOARD).
>
>       - Have all environment variables be terminal-local.
>         (Keep reading!) :-)
>
>       - Getenv should look at the current binding of
>         `process-environment' only.
>
>       - Modify the default behaviour of `setenv' to change the
>         variable on all terminals, one by one.  In a loop.

There is a lot of code that does the equivalent of

(let ((process-environment (copy-sequence process-environment)))
  (setenv "THIS" ...)
  (setenv "THAT" ...)
  (call-process "...")
)

It is the _standard_ way AFAICS to start a process with a custom
environment.  With your approach, this would not retain locality.

With my proposal, it would not work for setenv on any strictly
terminal-related variable (which would not make it into
call-process) unless you used (setenv-terminal "...

>         If it is desired that M-x setenv affect future terminals as
>         well, then we can make it keep a list of changed variables and
>         apply this list on the terminal-local environment whenever a
>         new terminal is initialized.  I suggest doing this.

This sounds too complicated for me.  I'd rather have a set of strictly
terminal-local and terminal relevant settings that will _override_
process-environment when processes are being called.

Keeping lists around to apply to other lists when terminals get
initialized: too complex.

>       - To support having only DISPLAY and a selected few other
>       variables differ on separate terminals, we can tweak the
>       initialization of terminal-local environment lists to copy the
>       rest of the variables from the original environment.  The
>       lists will, however, remain separate.

I don't like the implications for setting environment variables in
.emacs, for example.

> The above solution is easy to implement, easy to explain, doesn't
> involve fragile Lisp structures, and does not break any existing
> code.

I consider it fragile, and I don't see how you can claim it does not
break existing code when the most common idiom I cited above for
calling a process with temporarily set variables will not work.

> Existing Elisp code that does not change to a (literally) randomly
> selected frame after temporarily setting up a particular environment
> will work without changes when multiple terminals are simultaneously
> present in Emacs 23.

There is lots of Elisp code that does not even run in a frame: network
buffers, spell check buffers, background processes and the like.

> One deviation in the multi-terminal case is that code like
>
>       (let ((oldval (getenv "FOO"))
>         (setenv "FOO" "fred")
>         (unwind-protect
>               ...
>            (setenv "FOO" oldval))
>
> will change the value of FOO on all terminals but the current one as
> a new side effect.  This is, I believe, acceptable.  Some code
> adaptation to make existing code able to take advantage of the new
> feature set is inevitable.  We can simply document this concern and
> suggest solutions (e.g., let-binding `process-environment' instead,
> or adding a standard macro that would save and restore
> `process-environment' on all terminals.) in the NEWS file, or
> wherever we provide an upgrade guide for package writers.

Huh?  Let-binding process-environment would not keep your
setenv-implementation from iterating through the terminals and/or
keeping score of environment changes to apply, would it?

>> For some reason it would seem that
>
> (permanent)

Please don't "correct" things in my wording that are simply wrong.
The problem is exactly that also temporary changes of
process-environment are done using setenv (after copy-sequencing
process-environment).

>> manipulation of process-environment happens almost
>> exclusively through setenv.
>
> In the new design, this will work unchanged in multi-terminal
> sessions as well.

It won't for temporary changes.

>> So here are a few options:
>> 
>> a) make process-environment a terminal-local R/W variable.  Notice
>> that this does _not_ imply that anything but the first element of the
>> list can't be actually shared among the lists.  As long as
>> manipulation of process-environment happens with setenv, we are off
>> ok.  Changes of existing values can be done with setcar, so that
>> terminal-local environment variables (at the start of the list) will
>> stay terminal-local and vice versa.
>
> AFAICS, this is my above design, plus tail-merging, minus global
> setenv.  I don't hate it.  As I said above, if you'd really prefer,
> I can live with the shared tails, provided they can be easily
> disabled.

Well, I said that this idea has some hackish charm to it, but it
breaks down with regard to determining just where the shared tail
starts, how to initialize this for a new terminal and a few other
things.  I just mentioned it for its hack value, but I really don't
think we should work with such trickiness as a central design part.

>> b) make process-environment a global variable that can be manipulated
>> like the user wants.  However, whenever call-process or similar are
>> invoked, the actual environment passed into the called process has a
>> set of terminal-local environment settings prepended.
> [...]
>> It will be a nuisance and might break those programs that manipulate
>> the environment variables accessing the tty, but only those.
>
> I think this solution would be both incompatible and much too
> complex.

Interesting.  It is actually compatible with existing code (except
those setting the TERM variable if we agree that this is one of the
terminal-local ones), and it is quite simpler than your proposals.
The main disadvantage is that it has _one_ "Huh?" factor: setting a
terminal-related variable with setenv will not propagate to called
processes by default.

In contrast, your proposals have a _lot_ more "Huh?" factors.

> [...]
>> And it will also cause a lot of
>> inconsistencies that can't really be explained well to the user, like
>> "compile" behaving differently in windows that are side-by-side.
>
> I have explained above why I think this is an invalid argument with
> terminal-local variables.  If two windows are side by side, then they
> are on the same terminal.

I think the above comment belonged to a different model
(frame-dependent variables and/or complete environment
terminal/frame-local).

>>>       Terminal-local environments would less complete, but still
>>>       good enough to be usable without many problems.
>> 
>> That's what I would aim for, and only for those variables that are
>> indeed terminal-dependent.
>
> OK, so let's do that using my design.  Or yours.

If you can improve your design to a point where the
(let ((process-environment (copy-sequence process-environment)))
  (setenv "..."
  (setenv "..."
  (start-process ...
))

scenario accepts setenv for terminal-local variables without
propagating them elsewhere, where
(setenv "..."
alone will affect the environment everywhere (except where
terminal-specific variables are concerned) and there is not a lot of
background stuff going on that is hard to explain, then I would not
mind dropping my proposal.  It is just that at the current point of
time I don't see either your or mine proposal doing all that with a
reasonable amount of simplicity.

>> I think we should aim for the simplest solution that we can reasonably
>> explain.  My favorite solution, as explained above,
>
> ... is complex, hard to explain and backwards incompatible.

Funny.  That's exactly what I consider your solution to be.

> The new design can be basically explained in one short sentence:
> "process-environment is terminal-local, but setenv affects all
> terminals."

But we don't want a setenv DISPLAY to affect all terminals, and we
don't want it to affect any terminal if we are let-binding
process-environment.

> For example, I believe "environment variables are client-local, but
> setenv changes all clients at once" is a pretty useful description
> of current environment semantics on the branch.  No handwaving is
> necessary.

"client-local".  There are a lots of places inside of Emacs where
processes may get started that are a far way from being associated
with any "client".

>> Disagree.  "Similar packages" pretty much include _all_ packages
>> that would have reason to access the environment, so _certainly_
>> those packages are relevant to the issue.
>
> Um, nope.  M-x shell is special because it creates a long-term
> subprocess with which the user may communicate with inside Emacs,
> from different terminals.  X clients manually started from an M-x
> shell buffer will appear on the terminal that was active at the time
> the shell process was forked.  We can not change this fact, no
> matter how hard we tweak Emacs's environment variable handling.
> This is why I say the behaviour of M-x shell, M-x term, GUD, ILISP
> and friends (a.k.a.  "similar packages") is irrelevant to this
> discussion.

Again: disregarding most packages which actually access the
environment as "irrelevant" does not seem like a good idea.

> Packages starting x clients such as xdvi and short subprocesses such
> as compilations are, on the other hand, indeed relevant.

If they don't actually set the environment, they are not relevant for
discussing setting the environment.  They are only relevant for
establishing that we want some sort of locality for terminal-related
environment variables.

But there is agreement about that, so you don't need to cite them as
supporting your position in the unrelated matter of how to deal with
_writing_ to environment variables.

-- 
David Kastrup