Buffer-local process environments

[Augusto Stoffel]

When developing software, it's often not possible or desirable to
install compilers and related tools globally, so (barring the
availability of fancy build tools) one has to fiddle with environment
variables.

Currently it's quite hard to set up Emacs to interact correctly with
external tools on a per-project basis, because there is no unified way
adjust the environment.  For instance:

- `M-x compile' has the `compilation-environment' variable.
- python-mode uses `python-shell-virtualenv-root' when launching a
  REPL.
- Eglot provides no other mechanism than setting `exec-path' and
  `process-environment' buffer-locally.
- etc.

Of course, the buffer-local variable approach will work with any
command that doesn't switch buffers, even if that command invents its
own method to adjust the environment.

So why not make buffer-local process environments an official thing?
By this I mean:

1. Fix commands that start external processes after switching buffers
   to do what the user means.

2. Provide a convenient way to set the process environment buffer
   locally.

Point 1 is simple.  I've attached a patch for `compile' as an example.
In the Python REPL case, I'm not sure whether it would be better to
just adapt `run-python' or make a change in comint itself.

Proposal for point 2
--------------------

Add two more special keywords to the file local variable mechanism
(besides `mode', `eval', etc):

- `env' to add an entry to `process-environment'.
- `path' to add an entry to `exec-path' and modify the PATH variable at
  the same time.

Hence, the local variables section of a Python file might look like
this:

# Local Variables:
# path: "~/path/to/some/virtualenv/bin"
# env: "VIRTUAL_ENV=$HOME/path/to/some/virtualenv"
# env: "LANG=C"
# End:

Of course, in practice it's more useful to include this in
.dir-locals.el, but the mechanism is the same.

Closing remarks
---------------

As an indication that this is a real problem, let me point out a list
of MELPA packages just to deal with Python virtualenvs:

- auto-virtualenv
- auto-virtualenvwrapper
- conda
- pungi
- python-environment
- pyvenv
- virtualenv
- virtualenvwrapper

I've tried a couple of those and was never satisfied.

My proposal here is inspired by the `inherienv' [1] and `envrc' [2]
MELPA packages.  The former provides an advice to solve point 1, the
latter solves point 2 by means of an external program called direnv.

[1]: https://github.com/purcell/inheritenv/
[2]: https://github.com/purcell/envrc

>From ece87c73bcb5f8e2a59769cfe981c1486e19857b Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Thu, 29 Apr 2021 12:45:04 +0200
Subject: [PATCH] Make `compile' respect buffer-local process environment

* lisp/progmodes/compile.el (compilation-start): Use
`process-environment' from original buffer in the compilation process.
---
 lisp/progmodes/compile.el | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index 7a02c3a896..ee73f03c93 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1779,6 +1779,8 @@ compilation-start
            (replace-regexp-in-string "-mode\\'" "" (symbol-name mode))))
         (thisdir default-directory)
         (thisenv compilation-environment)
+         (this-process-environment process-environment)
+         (env-buffer (when (local-variable-p 'process-environment) 
(buffer-name)))
         outwin outbuf)
     (with-current-buffer
        (setq outbuf
@@ -1856,6 +1858,9 @@ compilation-start
                "; default-directory: "
                 (prin1-to-string (abbreviate-file-name default-directory))
                " -*-\n"
+                (if env-buffer
+                    (format "Process environment is local to buffer `%s'\n" 
env-buffer)
+                  "")
                (format "%s started at %s\n\n"
                        mode-name
                        (substring (current-time-string) 0 19))
@@ -1875,7 +1880,7 @@ compilation-start
               (and (derived-mode-p 'comint-mode)
                    (comint-term-environment))
              (list (format "INSIDE_EMACS=%s,compile" emacs-version))
-             (copy-sequence process-environment))))
+             (copy-sequence this-process-environment))))
         (setq-local compilation-arguments
                     (list command mode name-function highlight-regexp))
         (setq-local revert-buffer-function 'compilation-revert-buffer)
-- 
2.30.2