Re: Multi-tty design (Re: Reordering etc/NEWS)

[David Kastrup]

Karoly Lorentey <address@hidden> writes:

> No, my memory has failed me.  I had a patch implementing the above
> design, but what we currently have in the tree is something more
> complex: environment variables are neither frame-, nor
> terminal-local, but rather client-local.

I have seen on the archives of the multi-tty list and its README that
the implementation has went through several different ideas.  So quite
a bit of work has been invested already, and there is obviously not
going to be much enthusiasm for scrapping it.

However, at the current point of time the documentation seems to be
restricted to what is actually present in the code: it remains an open
task to write user level documentation (the Emacs manual) and
programmer level documentation (the Elisp manual).

This is actually a good thing because it provides sort of a litmus
test for interface usability and design quality: if one feels
uncomfortable casting existing behavior into a clear description and
instead is tempted to write something like "don't bother about it, it
will magically do the right thing", then something is amiss.

So we can first agree on what kind of interface we would not feel
ashamed putting into the manuals, and then adapt the code to it.

> The client-local environment list is shared between frames that were
> created in the same Emacsclient session.  The environment list is
> stored in a single frame's environment parameter; the other frames'
> environment is set to this frame.  (See `frame-with-environment'.)
> Frames not originating from an emacsclient session get the
> environment of the Emacs process itself, by the same
> mechanism. (Note that when the user exits emacsclient, all frames
> belonging to that client are automatically deleted.)

I think that "client-local" is a complication we really don't want to
introduce.  It is clear that frames on different servers or ttys will
have to have different personalities regarding the terminal, and in
particular regarding the exported value of DISPLAY to processes.  The
situation is less clear about values like TERM: however, exporting
them (or their equivalent) seems reasonable when we are talking about
MSDOS where a started subprocess will run in the tty of the actual
Emacs and talk to it bypassing Emacs' redirection stdout/stderr.

There is some sort of minor precedent to exporting a terminal-adapted
environment.  Namely we have

(defun comint-exec-1 (name buffer command switches)
  (let ((process-environment
         (nconc
          ;; If using termcap, we specify `emacs' as the terminal type
          ;; because that lets us specify a width.
          ;; If using terminfo, we specify `dumb' because that is
          ;; a defined terminal type.  `emacs' is not a defined terminal type
          ;; and there is no way for us to define it here.
          ;; Some programs that use terminfo get very confused
          ;; if TERM is not a valid terminal type.
          ;; ;; There is similar code in compile.el.
          (if (and (boundp 'system-uses-terminfo) system-uses-terminfo)
              (list "TERM=dumb" "TERMCAP="
                    (format "COLUMNS=%d" (window-width)))
            (list "TERM=emacs"
                  (format "TERMCAP=emacs:co#%d:tc=unknown:" (window-width))))
          (unless (getenv "EMACS")
            (list "EMACS=t"))
          (list (format "INSIDE_EMACS=%s,comint" emacs-version))
          process-environment))

However, this holds for just a particular environment provided by
Emacs itself.

In a similar vein, we have
(defun term-exec-1 (name buffer command switches)
  ;; We need to do an extra (fork-less) exec to run stty.
  ;; (This would not be needed if we had suitable Emacs primitives.)
  ;; The 'if ...; then shift; fi' hack is because Bourne shell
  ;; loses one arg when called with -c, and newer shells (bash,  ksh) don't.
  ;; Thus we add an extra dummy argument "..", and then remove it.
  (let ((process-environment
         (nconc
          (list
           (format "TERM=%s" term-term-name)
           (format "TERMINFO=%s" data-directory)
           (format term-termcap-format "TERMCAP="
                   term-term-name term-height term-width)
           ;; We are going to get rid of the binding for EMACS,
           ;; probably in Emacs 23, because it breaks
           ;; `./configure' of some packages that expect it to
           ;; say where to find EMACS.
           (format "EMACS=%s (term:%s)" emacs-version term-protocol-version)
           (format "INSIDE_EMACS=%s,term:%s" emacs-version 
term-protocol-version)
           (format "LINES=%d" term-height)
           (format "COLUMNS=%d" term-width))
          process-environment))

The situation for emacsclient is somewhat different because we want to
have _everything_ affected that gets called via either call-process or
start-process.

Here is how to approach this with a "minimally invasive hack" (namely
something which works with most existing code, but which one would not
really want to ever write into a manual): make process-environment a
terminal-local variable.  It will be an nconc of terminal local
environment variables and `global-process-environment', the
environment Emacs got started with.  setenv uses setcar to manipulate
existing environment variables, and appends (instead of prepends) them
to process-environment when they don't yet exist.  That way,
environment changes will be global unless a terminal-local environment
variable already exists.

Ok, this is just not feasible: it is too much of an ugly hack (lists
with shared tails are not nice to understand, and when the heads are
stored in terminal-local variables, we gain an additional level of
ugliness).

So we have to see how we are actually going to work with this and take
a look at existing code.  It turns out that quite a few functions
let-bind process-environment to something nconc-ed together starting
with current value of process-environment.  There are also several
times where first process-environment is let-bound to a copy-sequence
of its previous binding, then setenv is being applied in order to
manipulate this copy, and finally a process gets started while the
let-binding is still in effect.

Also, currently the _default_ way to search through the current
environment without knowing what to look for in advance, is to use
process-environment.

If we take a look at how setenv/getenv/process-environment are
actually used, some patterns emerge:

While getenv and looking through process-environment are more or less
used interchangeably, we have (in non-multitty Emacs) very few
occurences where process-environment actually gets written to:

find /rep/emacs -type f -name \*.el -print0 | xargs -0 -e grep -nH -e "(setq 
process-environment"
/rep/emacs/lisp/env.el:152:                (setq process-environment (delq (car 
scan)
/rep/emacs/lisp/env.el:159:           (setq process-environment
/rep/emacs/lisp/startup.el:398:          (setq process-environment

And that is it.  In all other cases, it is usually let-bound to a copy
of itself, or something nconc-ed or concatenated to in front of
itself.  This can occur in combination with setenv.  When a
let-binding is employed together with copy-sequence and setenv, the
expectation is that the effect will be completely temporarily.

So those are the semantics we want to keep if we possibly can.  We
also would want to have terminal-local environment variables (like
"DISPLAY") appear in process-environment.  For some reason it would
seem that manipulation of process-environment happens almost
exclusively through setenv.

So one solution will be: have process-environment _reflect_ the
current environment but not determine it.  There is a problem with
that: letting current-environment to a copy of itself is supposed to
make setenv register changes, but only temporarily.

So here are a few options:

a) make process-environment a terminal-local R/W variable.  Notice
that this does _not_ imply that anything but the first element of the
list can't be actually shared among the lists.  As long as
manipulation of process-environment happens with setenv, we are off
ok.  Changes of existing values can be done with setcar, so that
terminal-local environment variables (at the start of the list) will
stay terminal-local and vice versa.

Disadvantage: if somebody manipulates process-environment with setq
and a copy, the variable set for this terminal will _then_ become
completely detached from that of other terminals.  Which is actually
what Károly would prefer...

b) make process-environment a global variable that can be manipulated
like the user wants.  However, whenever call-process or similar are
invoked, the actual environment passed into the called process has a
set of terminal-local environment settings prepended.  This means that
M-x setenv RET DISPLAY RET will be reflected in the
process-environment of all terminals, but actually called processes
will have a different setting.  To mitigate the unintuitiveness of
this approach, setenv could also, where appropriate, record the
setting into the terminal-local settings which default to be the
active value at the terminal, but could be shadowed for this purpose
so that setting them becomes possible.  However, this would mean that
setenv used on a let-bound process-environment would, after all, leave
a lasting effect on the environment.

So one should really have to use a special setenv variant to set the
terminal-specific overriding environment: the normal setenv should not
go there.

For that reason, we want to keep the extent of the terminal-specific
environment to a bare minimum.

If people have to learn to use special commands or command variants
for setting _those_ (and only those) environment variables intimately
connected with the tty, I suppose we can make the issue understood.

It will be a nuisance and might break those programs that manipulate
the environment variables accessing the tty, but only those.

We can provide a warning if people use setenv on such variables
interactively.

>       - For the user, there is a strong sense of connection between
>       an emacsclient instance and its set of frames.  If emacsclient
>       exits, then its frames are deleted and vice versa.
>
>       C-x C-c kills emacsclient, not the entire Emacs process.  All
>       this feels very natural and fits a range of common use-cases,
>       particularly the ones involving quick editing jobs from the
>       command prompt.  (These are the ones for which Emacs was so
>       infamously not well suited before.)
>
>         This means that having a different set of variables from frame to
>         frame does not normally seem inconsistent or unpredictable to the
>         user.

I am opposed to claiming to create an illusion that we can't actually
provide.  kill-emacs should work as before.  However, one might
consider providing a different command kill-terminal-group or similar
which would instead be bound to C-x C-c.

>       - Furthermore, to me it seems more consistent to have all
>       environment variables be local than just a select few of them.

But it will pretty much break every Lisp program involved with the
environment.  That price is too high.  And it will also cause a lot of
inconsistencies that can't really be explained well to the user, like
"compile" behaving differently in windows that are side-by-side.


>       - Client-local environments mean that two separate emacsclient 
> instances          on
> the same terminal can have different environments:
>
>               $ FOO=23 emacsclient
>                       (getenv "FOO") ==> 23
>               C-z
>               $ FOO=42 emacsclient            # Other frame is still alive,
>                       (getenv "FOO") ==> 42   # with a value of 23.
>
>           This is a corner case that deepens the illusion of emacsclient being
>         a real standalone editor.

But since we can maintain that illusion at a shallow level at best and
since it will get in the way of trying to actually access variables,
it is not a good idea to aim for it.

>         It is not an important feature, although it did often come
>         useful for me, particularly with long emacsclient sessions
>         for different tasks, but sharing a single X display.

I don't see how.

>         Terminal-local environments would less complete, but still
>         good enough to be usable without many problems.

That's what I would aim for, and only for those variables that are
indeed terminal-dependent.

>       - The behaviour of M-x shell and similar packages is mostly
>       irrelevant here.

Disagree.  "Similar packages" pretty much include _all_ packages that
would have reason to access the environment, so _certainly_ those
packages are relevant to the issue.

> If you find client-local environments unacceptably ugly, I can
> update and submit my patch for simple terminal-local environments.
> The local environment then becomes a simple terminal parameter,
> initialized when the terminal is created.  This is a much simpler
> solution that retains much of the feature set of the current design.

I think we should aim for the simplest solution that we can reasonably
explain.  My favorite solution, as explained above, still has the
disadvantage that setenv in a single-terminal environment will still
behave differently from before for a strictly limited set of
variables.  I don't see how we could avoid this while maintaining
reasonable upwards compatibility.

-- 
David Kastrup