Re: Core ELPA was: Testing fontification, indentation, and buffer manipulation

[Phillip Lord]

Stefan Monnier <address@hidden> writes:

>> Wondering whether anyone had time to give me feedback on this. It
>> provides a mechanism to have ELPA packages bundle with core Emacs.
>
> Sorry for taking so long.  I just took a look at it, and it looks fairly
> simple and clean.  But I think the main question is *if* we really want
> to use this to build the official tarballs.  I think we should, but IIRC
> Eli wasn't quite as enthusiastic about it.

Indeed. It also means that git will (sometimes) be needed to build;
although this will only be true after git pulling the main git repo, so
perhaps this is not such an issue.

I would also plan to build two tar balls -- emacs-and-selected-elpa.tgz
and emacs.tgz. Over time, I think it would make sense to change these to
emacs.tgz and emacs-minimal.tgz (i.e. with ELPA becomes the default). 


>>  - Doesn't use package.el (I think this is an anti-feature, but tried
>>    that before and no one liked it)
>
> That doesn't ring a bell.  Could you refresh our memory why no one
> liked it?

A variety of reasons. It's less simple. It requires changes to
package.el (for example, what should emacs -Q do with ELPA packages. In
my original version they would be disabled, in this version they are
still on). package.el has to be launched every start up for every
emacs. And Eli in particular didn't like the increase in the number of
directories in the default load-path. This implementation does the same
thing, but it doesn't have to -- we could copy all the lisp files into
the same directory.



>> --- /dev/null
>> +++ b/notes.org
>> @@ -0,0 +1,84 @@
>> +
>> +
>> +* Working on
>> +
>> +Have package checkout working for external
>> +Next get it working for non external so we can template
>
> Don't know what you mean by "can template"

Sorry the notes are a bit out of date (bad habit of mine, and I wasn't
expecting anyone to read this file, cause it will go before merge).

This is working now (neither queue nor darkroom are external).


>> +* To Fix
>> +Think we are running git fetch --all on every package.
>
> Don't know what you're referring to here either.


Also fixed. Getting the dependencies was a bit tricky. 


>> +1) Scripting?
>> +Guess we have to use sh
>
> Of course, we could also use Elisp ;-)

If I remember correctly, all of this happens before the full Emacs is
dumped. It seemed cleaner to do this to me.


>> +2) How to get specific directory of a repository
>> +
>> +
>> +
>> +git archive
>> address@hidden:/srv/git/emacs/elpa.git
>> master packages/hydra | tar xv --strip-components=1
>> +
>> +Currently, this requires a member login -- it doesn't work with http
>> +or the current git server.
>> +
>> +Would also be possible to configure this from local.
>
> I'm not familiar with `git archive` but does this require network
> access?

git archive gives a bare, not with a .git, directory.

I had to drop that command anyway. Git doesn't allow checkout of an
arbitrary SHA from a remote repo (because it would allow checking out
commits that had been deleted otherwise, I believe). So instead, now,
the makefile checkouts out the whole of ELPA, and then uses git archive
to get a specific SHA from local.

Obviously, this is a fairly network intensive way of getting just a few
files; but it's a fixed sized problem.


>> +3) How to work out when a directory is out of date wrt to the make
>> +   file
>
> Don't know what you mean here.


That's fixed also. Packages should updated from git (and git fetch run)
if Makefile is updated. So, this means a git fetch after everytime Emacs
is ./configure is run.

>
>> +4) Heuristics for working out which files to copy to right place
>> +
>> +*-test.el
>> +*-tests.el
>> +/test/*.el
>> +
>> +to test
>> +
>> +All other *el to lisp
>> +
>> +Nothing else
>
> Rather than heuristics, I'd go for a *rules* (which needs to be obeyed
> before the package can be bundled).

ELPA already uses heuristics (see admin/ert-support.el). Rules would
make this simpler, of course, at the cost of, well, another rule.


> But I'm not sure we should install those packages with a different
> layout than for a "normal" ELPA install.
> What's the advantage of using a different layout?

I haven't checked to see who uses them or why, so I don't know.

>
>> +6) How to enable ELPA packages to customize the process
>> +Add the option to run an core-deploy.sh file placed into the root of a
>> +package.
>
> Don't know what you mean here.


That bits out of date, and supported another way.

By default, EMACS/elpa/bin/package-makefile.mk is copied into a package
after it is pulled out of git and it is this that copies files from the
elpa directory into the core Emacs layout. If a package wants to
override this behaviour, it could include it's own package-makefile.mk.


>> +8) Support other repos
>> +Org-mode effectively does this already
>
> Don't know what you mean here.

Currently, this will only take files from the ELPA repo, but there is
not reason that this needs to be so. Files could come another repo (such
as org-mode's own). My guess is that there is no particular reason to
support this at the moment.


>> --- /dev/null
>> +++ b/elpa/bin/extract-package.sh
>> @@ -0,0 +1,55 @@
>> +#!/bin/bash
>> +
>> +function grab_external {
>> +    rm -rf packages/$PACKAGE*
>> +    mkdir --parents $PACKAGE_LOC
>> +    cwd=`pwd`
>> +    cd $GIT_LOC
>> +    git archive $SHA \
>> +        | tar xv -C $cwd/$PACKAGE_LOC
>> +    cd $cwd
>> +    cp --no-clobber bin/package-makefile.mk $PACKAGE_LOC
>
> Rather than `cd $cwd`, I generally prefer to use (...) to save&restore
> the current directory:
>
>     cwd=`pwd`
>     (cd $GIT_LOC
>      git archive $SHA \
>          | tar xv -C $cwd/$PACKAGE_LOC)
>     cp --no-clobber bin/package-makefile.mk $PACKAGE_LOC

Oh, yes, hadn't thought of that.


> If we keep the layout unchanged, we could use `git worktree add`, which
> has the advantage that when running Emacs in-place, jumping to the
> source in `C-h f` and friends gives you access to the Git metainfo.

Not sure I have understood -- you mean git subtree add perhaps? This
wouldn't work anyway, because C-h f will jump to source in
EMACS/lisp/elpa which is not created by git archive.

Phil