Re: user-directory: New library to find user {conf, data, state, ...} files

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Sun, 7 Nov 2021 21:04:42 -0600
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Note that on GNU/Linux, even if the user has their configuration file in
> >> "~/.emacs.d/", the library currently prefers the XDG paths here:
> >>
> >>     (user-file 'cache "image-dired")
> >>     => "~/.cache/emacs/image-dired"
> >
> > This contradicts how Emacs by default finds the init file: its
> > preference is the other way around, and it sets user-emacs-directory
> > according to the directory from which it loaded the init file.  I
> > don't think I understand why this package should behave differently by
> > default (it could do that as an opt-in option, of course, if that is
> > considered useful to someone).
> 
> It should because it is a change for the better.  We have a chance to
> decide what this should look like for the foreseeable future, and I
> don't think we are bound by past decisions if they do not produce
> optimal (or even good) results.

The problem is that changes which break users' configurations will not
be appreciated.

> The `user-emacs-directory' directory currently contains a grab-bag of,
> well, everything, which among other things makes it hard for a user to
> know what is safe to delete and what is not, which files you would need
> to backup, etc.  You basically need to manually inspect the files and
> consider their content one by one.
> 
> The big use-case here, though, is that we want to be able to easily
> synchronize our Emacs configuration file between several machines.  You
> don't want to synchronize things like your completion history or
> recently opened files, because it is not very useful, and you will have
> an infinite amounts of conflicts.  (It's somewhat easier with version
> control, which is what I use, but less so if you want to use rsync or
> unison.)
> 
> The solution is to proscribe that:
> 
> The categories data, cache and state files belong in specific `data',
> `state' and `cache' directories.  This means that e.g. my bookmarks file
> goes to the data directory, the recentf and saveplace file goes to
> `state', while my image-dired thumbnails go to `cache'.  As for runtime
> files, I note that `server-socket-dir' already uses XDG_RUNTIME_DIR (but
> gets it manually).
> 
> IOW, all this should be orthogonal to where your init file is placed.
> You can put it in any of the accepted locations, including the XDG one
> if you like.  (Personally, I keep mine in "~/.emacs.d/init.el" and will
> continue to do so.)  You should still be able to get all of the above,
> by default and with no hassle.

My problem is that Emacs computes user-emacs-directory from the place
where it loaded the init file, and then uses the value of
user-emacs-directory to look for other files.  Search the sources for
user-emacs-directory and you will see how we use it.  If the changes
you propose will modify those users of user-emacs-directory to use
user-file machinery instead, and if user-file machinery prefers the
XDG directories, Emacs will not find the files it found previously,
and some of the user customizations will mysteriously stop working.

> With all that being said, we must take great care not to cause
> unnecessary churn.  In particular, we should absolutely not break
> anything that's currently working.  This means that we should respect
> locations of already existing files, and ensure they continue working
> seamlessly.

That is fine, but I'm not sure how you can achieve that goal and still
prefer the XDG directories.

> PS. Note that (user-directory 'config) in particular already just falls
>     back to use whatever `user-emacs-directory' was set to.  There is no
>     need to duplicate what is happening in startup.el, or to try to
>     outsmart it.

What do you mean by "falls back"?  To respect the current behavior,
the value of user-emacs-directory should be used in preference to
everything else.  But if we do that, then in which cases will the XDG
directories be used, since user-emacs-directory always exists and is
defined.