bug#45316: [PATCH]: Re-introduce Emacs packages specific installation prefix.

[Leo Prikler]

Hi Maxim,
Am Samstag, den 26.12.2020, 23:44 -0500 schrieb Maxim Cournoyer:
> Hi Leo,
> 
> Leo Prikler <leo.prikler@student.tugraz.at> writes:
> 
> [...]
> 
> > > If you want to have a clearer idea of how packages from ELPA and
> > > the
> > > likes are installed, you can have a peek into the 'package.el'
> > > file
> > > shipped with Emacs (spoiler: it's basically just extracting the
> > > contents
> > > of the package archive to a single directory -- see the `package-
> > > unpack'
> > > procedure).
> > If we follow that to the letter, should we not install those
> > packages
> > directly to site-lisp/$PACKAGE-$VERSION?  Why the guix directory?
> 
> I used to wonder the same, but after getting familiar with the
> 'package'
> Emacs library, it defaults (per the default value of the
> `package-directory-list' variable) to such a layout:
> 
> directory-on-load-path/prefix/name-version/
> 
> where 'prefix' defaults to 'elpa'.  It seems a good idea to have the
> ELPA-style packages live by themselves, separated from the 'classic'
> packages that are installed directly to the load path, to avoid the
> package.el loader getting confused or scanning entries it wouldn't
> know
> how to load anyway.
> 
> That said, it's shouldn't be strictly required, as old style packages
> or
> resource directories would be lacking a *-pkg.el file and would be
> disregarded, and it should be possible to install the package
> directories directly to the load path if we wanted to.
I don't really know how to feel about faking elpa.  Also, I don't think
that 'classic' means 'not supported by package.el'.  There are other
reasons to use e.g. gnu-build-system over emacs-build-system and
packages might ship a *-pkg.el even if they do build using a Makefile. 
It's also trivial to provide one if they don't and Guix has the
facilities to do so.

> > Also, if we do go that route, I think we should install a
> > subdirs.el
> > through a profile hook, that adds all these package directories to
> > the
> > load path.  See also '(elisp) Startup Summary'.  Alternatively, we
> > can
> > let subdirs.el defer to guix-emacs.el, but that'd be very cheeky.
> > Either way, we should definitely make -Q work.
> 
> If we use a subdirs.el file, it could add the ELPA-style packages
> more
> simply if they are under the guix sub-directory (guix/*).  Otherwise
> we'd have to filter the resource directories that potentially find
> their
> ways to the load path for old style packages.  Our custom subdirs.el
> file could be shipped with our Emacs packages, installed in their
> output
> at share/emacs/site-lisp/subdirs.el.  When a profile union would be
> created this subdirs.el would automatically be made available.
IOW if we don't use guix/*, we would have to at once fix all our
packages, that don't adhere to this standard.  Sounds like a bit of
trouble, but also like it'd improve consistency.

> > > Here's an experiment I've done with a profile containing the
> > > whole of
> > > our current emacs-build-system based packages, with this current
> > > change,
> > > that checks for collision at the top level of a package
> > > installation
> > > directory:
> > > 
> > > --8<---------------cut here---------------end--------------->8---
> > > $ ./pre-inst-env guix package -p /tmp/nnew-emacs -m emacs-
> > > packages-
> > > manifest.scm
> > > 
> > > $  find -L /tmp/nnew-emacs/share/emacs/site-lisp/guix -maxdepth 2
> > > -mindepth 2 -not -regex '.*\.elc?$' \
> > >           | awk -F '/' '{ if ($9 in packages) \
> > >             {printf("%s directory of %s package collides with
> > > that of
> > > package %s\n", $9, $8, packages[$9])} \
> > >             else {packages[$9] = $8} }'
> > > 
> > > doc directory of modus-operandi-theme-1.0.2 package collides with
> > > that of package racket-mode-0.0.2-6.5eb31a2
> > > tools directory of unidecode-0.2-1.5502ada package collides with
> > > that
> > > of package company-cabal-0.3.0-1.62112a7
> > > snippets directory of minitest-0.8.0-1.1aadb78 package collides
> > > with
> > > that of package elpy-1.35.0
> > > test directory of realgud-1.5.1 package collides with that of
> > > package
> > > flycheck-haskell-0.8-2.32ddff8
> > > snippets directory of yasnippet-snippets-0.23 package collides
> > > with
> > > that of package elpy-1.35.0
> > > snippets directory of haskell-snippets-0.1.0-0.07b0f46 package
> > > collides with that of package elpy-1.35.0
> > > snippets directory of rspec-1.11-1.66ea7cc package collides with
> > > that
> > > of package elpy-1.35.0
> > > doc directory of modus-themes-1.0.2 package collides with that of
> > > package racket-mode-0.0.2-6.5eb31a2
> > > data directory of emojify-1.2 package collides with that of
> > > package
> > > unidecode-0.2-1.5502ada
> > > test directory of systemd-mode-1.6 package collides with that of
> > > package flycheck-haskell-0.8-2.32ddff8
> > > lib directory of slime-2.26.1 package collides with that of
> > > package
> > > robe-0.8.2
> > > contrib directory of sly-1.0.0-7.68561f1 package collides with
> > > that
> > > of package slime-2.26.1
> > > lib directory of sly-1.0.0-7.68561f1 package collides with that
> > > of
> > > package robe-0.8.2
> > > doc directory of modus-vivendi-theme-1.0.2 package collides with
> > > that
> > > of package racket-mode-0.0.2-6.5eb31a2
> > > doc directory of evil-1.14.0 package collides with that of
> > > package
> > > racket-mode-0.0.2-6.5eb31a2
> > > data directory of all-the-icons-4.0.1 package collides with that
> > > of
> > > package unidecode-0.2-1.5502ada
> > > snippets directory of feature-mode-20190801-1.11ae167 package
> > > collides with that of package elpy-1.35.0
> > > --8<---------------cut here---------------end--------------->8---
> > > 
> > > So 17 Emacs packages in Guix currently conflict, and Guix seems
> > > to be
> > > silent about it when building a profile with them via 'guix
> > > package
> > > -p'
> > > (a bug?).
> > It's a bug in those packages; not in Guix.
> 
> I used to think the same, but after reading the Elisp reference
> manual
> mentioning how Emacs packages to be installed with package.el are
> expected to live in their own distinct directory with their own
> resources; we can't really say that's it's a bug in the packages:
> they
> just weren't designed to be merged in a flat directory with other
> Elisp
> packages.  Users installing these packages manually can simply add
> their
> top level directory to their load path, else use package.el to
> install
> them.
Idk, feels like a mixed bag to me.  Especially yasnippet will actually
be broken by going to this new layout.  The user will be expected to
have $GUIX_PROFILE/share/emacs/site-lisp/yasnippet-snippets-
$VERSION/snippets in their yas-snippets-dir.  My really old setup
assumes, they'd be stored in $GUIX_PROFILE/share/emacs/yasnippet-
snippets/ instead, because I forgot it moved to site-lisp/snippets. 
Now what do you think happens on version upgrades?

I think, this is another argument to separate code and data even if we
do go the route of using subdirectories to store code.

> > I think I've already laid that out above, but we really ought to
> > have
> > site-lisp/$PACKAGE-$VERSION and a working subdirs.el.  If we do it
> > like
> > that, I don't think the multi-directory layout should cause us too
> > many
> > troubles.
> > 
> > As far as the build system is concerned, we would probably need to
> > set
> > up an environment similar to what will be found in the end.
> > Imagine this:
> > 
> > - environment-variables
> > - $PACKAGE-$VERSION/
> > - $PACKAGE-$VERSION-inputs/
> >   - subdirs.el
> >   - $INPUT1/
> >   - $INPUT2/
> 
> In the build environment, there is no profile so each package add
> their
> individual site-lisp/ to the load path (EMACSLOADPATH).  With my
> proposed idea to add subdirs.el to Emacs itself, there'd be nothing
> to
> do here.
Yes there would.  Exactly because there is no union-build, those
packages would not not have subdirs.el, so it's dubious as to whether
they'd be properly expanded.

> Thanks again for this discussion; I'll work on a revised
> patch.  Until
> then, happy new year!
Happy new year to you too and happy hacking 🙂

Leo