Re: GNU ELPA and package.el

[Achim Gratz]

Stefan Monnier writes:
> I'm fully aware of that.  But you can push the prebuilt packages to
> elpa.it rather than pushing them to some third-party distribution site,
> and let elpa.gnu.org build the corresponding tarball.  This is the only
> thing needed to eliminate the "Org hack".

So the only thing you care about is that you don't want to deal with the
tar file?  The reason Org gives you the tar file is that we already had
it available already (Org was distributing its own ELPA packages for
quite some time already), plus we'd rather have ELPA serve the exact
same tarball as you can get from Org's own site.

Let me discuss this on the orgmode list and I'm sure we can find a
solution.

> This said, I'd love to get some help improving the elpa.gnu.org build
> scripts so we can lift those restrictions altogether.
> [ Basically, I'd like to be able to run those package-provided extra
>   build rules in some kind of sandbox, e.g. an LXC container; but given
>   the limited resources of elpa.gnu.org and its maintenance, it should
>   ideally re-use the Debian install already provided by elpa.gnu.org.  ]

Building the release tarballs on the Orgmode server (which used to be a
very small VM on a shared box) didn't take all that much resources, but
a fair bit of time depending on how much the machine was loaded from
other tenants.  Nobody's waiting on the build to finish so she can use
it, so that works OK.

I seem to remember some discussion about Gitlab for GNU.  A Gitlab
instance would usually provide CI pipelines that could produce release
tarballs easily.  If that's on the horizon I'd rather not muck about
with building on the server (you are running code provided from external
sources after all) or inventing your own CI just so you can do it a
little more safely (however safe you think your hand-built solution is).

>> The other unsolved problem is that anything that gets built in to Emacs
>> releases still can't be later cleanly updated as a package because none
>> of the "built-in packages" are actually packages in the ELPA sense.
>
> I don't know what problem you're thinking of.
> You can definitely upgrade Org, python.el, and several others.
> I can imagine scenarios where it can partly break, but most of them seem
> contrived to me, so if you know of practical problems, please let
> me know.

The problem auf autoloads not being able to get redefined in a running
instance of Emacs.  I posted example code, a horrible hack of cleaning
the data structures manually and we've discussed if it was possible to
start a "clean" Emacs for byte compilation to work around this.

The problem was exacerbated by the fact that some Org autoloads would
end up in the global loaddefs.el, a problem that has since been fixed
IIRC (but can creep in any time again).

>> Last but not least I'll mention again that even if the two points above
>> were solved, there still is not mechanism to cleanly separate packages
>> installed at the system level (either with the Emacs release or
>> separately by the admin) and user-level packages.  Specifically, if
>> packages are installed at the system level, the user can either use them
>> all or none of them, but can't really chose on a per-package basis
>> (without jumping through a number of burning hoops, that is).
>
> Currently, this choice is made via package-load-list.  That was the
> initial way to do that, and there hasn't been much noise about it since,
> so it hasn't seen much activity, but we could make it more flexible
> if needed.  How do you imagine the user making this choice?
> You mean you'd like to have a "clickable UI" or something else?

Well yes, most users that can't install their own packages, but can use
package.el would be at loss to do it via package-load-list, but are not
expected to have problems if package.el told them which versions of a
package are avilable on their system and where and which of thos they
want to use (or install one from a package archive).  The other thing
package.el doesn't do at the moment is to "delete" a package that is
either built-in or installed system-wide without replacing it with a
user-installed (later) version.

>>> What you can do is create a Tramp package on elpa.git and push releases
>>> there (complete with the pre-built auxiliary files).
>> Well, that'd be more or less the same hack as you use for Org, except
>> you use Git instead of an archive file.
>
> That's a big difference for the build scripts.

See the beginning of this post.

>>> This is what AUCTeX does, basically (where the files that would
>>> ideally be auto-generated during packaging are instead stored in the
>>> elpa.git repository after making them manually).
>> That is a mistake and should not be forced on anyone.
>
> I don't consider it a feature, but other than complaints, I haven't
> gotten much help in trying to improve the situation.

The last time I tried to discuss the requirements of moving this along
(this really ties into so many places in Emacs that hopefully we have
clear specifications of what to implement before actually starting it)
it was either falling on deaf ears or producing almost allergic
reactions.  Not blaming anyone for not wanting to add to their to-do
list, but I didn't see a way forward and dropped it.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves