bug#48994: 28.0.50; [PATCH] 28.0.50; Native compilation unnecessarily re

On Sat, Jun 19, 2021 at 12:04 PM Alan Third <alan@idiocy.org> wrote:

On Wed, Jun 16, 2021 at 09:45:50PM +0300, Eli Zaretskii wrote:
> > Date: Wed, 16 Jun 2021 19:25:14 +0100
> > From: Alan Third <alan@idiocy.org>
> > Cc: 48994@debbugs.gnu.org, mjbauer95@gmail.com, akrl@sdf.org
> >
> > > My primary worry is not the Makefiles, it's what the installed Emacs
> > > does upon startup to find its requisite files.
> >
> > I've done a lot of digging about and it looks like a bit of a mess,
> > frankly.
>
> It could be. But let me first ask you: did you understand how Emacs
> finds the directory with the *.eln files, and how that is related to
> finding the .pdmp file? These two parts are looked for together, and
> the place where we find the .pdmp file tells us directly where to look
> for the *.eln files.
>
> If this is "messy" on macOS, in either of the two types of install,
> then we should fix that first.

I think I did understand that. As far as I can tell that bit actually
works fine, as long as the pdmp file can be found the eln files can be
found.

> So we need to make NS-specific changes that will make this work
> correctly both in Unix-style and the app bundle style install. Do you
> understand how to fix this part? I'll gladly help you if needed.

I've attached a patch which, I hope, should solve these problems. I've
tried to align the code more with how other terms work, so I've
removed all the functions that calculate the various paths Emacs uses,
and replaced them with a single function that, given a relative path
inside the application bundle, can work out what the full path should
be, but will pass any absolute paths unchanged.

It's similar to w32_relocate, but not exactly the same, and I'm using
it in more places due to the app bundle layout being quite different.

I've also followed the w32 lead in changing how epaths.h is generated.
Basically I post-process it so that any paths that are internal to the
app bundle are converted to relative paths, so the above mentioned
function can deal with them.

> > > . Emacs binary in /usr/bin
> > > . the .pdmp file in /usr/libexec/emacs/VERSION/ARCHITECTURE
> > > . the *eln files in /usr/lib/emacs/VERSION/native-lisp
> > >
> > > ?
> >
> > Sorry, I must've missed it.
> >
> > Yes, a normal Unix style install should be like that.
>
> OK, then the Unix style install should already be working. Then I
> wonder how come this very install caused the OP trouble. But we can
> revisit it later, once the code whch looks for .pdmp and *.eln files
> is finalized.
>
> > However the OP appears to be using a Unix style install with a
> > different install prefix and is getting app bundle install paths in
> > his errors, specifically Contents/Resources/lisp, which I mentioned at
> > the top of the email.
>
> I wonder how this could happen, if the Unix style install is supposed
> to "just work".

I think it's because of some hard-coding of things that was added to
work around problems in the app bundle code. They haven't been
suitably separated from the Unix style install code, so they end up
interfering with each other.

I think that now a Unix style install shouldn't be affected at all by
the app bundle code, but I'd like some extra testing by people who run
it that way before I'm fully convinced I've been successful in that.

> > > > I believe there's a ./configure flag. But like I said before, for the
> > > > other paths there's a run-time check, so I'm not sure how it all ties
> > > > together.
> > >
> > > Which run-time check did you have in mind?
> >
> > ns_exec_path, and friends. They return the app bundle paths if the
> > directories exist, and nil otherwise, then the code that calls them
> > modifies the search paths depending on the result.
>
> It's okay to use that, although I'd expect the install location to be
> known at compile time?

The install location is known, but in the app bundle case the app can
be copied to anywhere and then the full paths will be different, so
they can't just be hard coded at install time.

> > I think I can probably fix this now, hopefully without breaking
> > anything... ;)
>
> Fine, please do, but let's try doing that one step at a time...

I tried, but it ended up quite a large patch as fixing one thing would
invariably break another, and I decided I had to sort out the epaths.h
situation before I could do much else, and most of the rest of the
patch is dealing with the fallout from that.

Anyway, it's attached. It's working here for the app bundle case and I
believe it should work well for the Unix style install, but I've not
tested that in any depth.

Matthew, could you please try the attached patch and see if it solves
your problems (or makes them worse!)?
--
Alan Third

From:	Matthew Bauer
Subject:	bug#48994: 28.0.50; [PATCH] 28.0.50; Native compilation unnecessarily recompiles .eln (macOS)
Date:	Mon, 21 Jun 2021 22:55:40 -0500