[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#47180] [PATCH] gnu: racket: Don't inject store paths into Racket fi
[bug#47180] [PATCH] gnu: racket: Don't inject store paths into Racket files.
Sun, 11 Apr 2021 23:40:59 -0400
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.9.0
On 4/10/21 4:59 PM, Ludovic Courtès wrote:
Philip McGrath <firstname.lastname@example.org> skribis:
Apparently, during grafting, Guix can somehow mangle compiled
Racket CS files (.zo) such that Racket will refuse to load them.
(Maybe it has something to do with compression?)
If those files are compressed, and if a store file name survives despite
compression, then grafting can patch it, which could lead to checksum
mismatches or similar.
Yes, that's what seems to be happening.
What error message does Racket produce?
The first error I heard of (and reproduced) was reported by Jack in
$bytevector-uncompress: internal error uncompressing
The error message is referring to an internal Chez Scheme primitive.
The report at <https://issues.guix.gnu.org/47614> seems to me to be an
alternative manifestation of the same problem: the hexdump there is
useful and explains why I did see some of the store path when I had
attempted to investigate using `strings`.
So there seem to be at least three bad cases:
1. The grafter can mangle .zo files so that Racket can't
read them at all.
2. The grafter can miss store references in .zo files, so Racket could
end up using the ungrafted versions.
3. With a garbage collection, Racket could try to use the ungrafted
versions but fail to find them at runtime.
So, we stop patching Racket sources with absolute paths to store
files (i.e. for foreign libraries to dlopen).
Instead, we put them in a data file that doesn't get compiled or,
in one case, embed it in C.
That solves the problem for Racket itself, but wouln’t Racket libraries
have the same issue?
Because the problems are triggered by grafting, they only affect
packages that have been patched by Guix. For now, Guix doesn't have the
ability to install more Racket packages. In the longer term, the
one-sentence answer is that it's always possible we might find and
switch to a better approach, but this seems most promising. (A bit more
of my current thinking on that toward the end.)
Would it be an option to instead turn off compression and keep doing
things as usual?
In theory, this should be possible. I see two significant downsides:
1. Compiled code would be much larger—maybe twice as big—and, if I
recall correctly, load times would be worse, too. With the move to
Racket CS, existing Racket code moved from a world of small and
cheap bytecode to a world of machine code: the default compression
settings have been tuned to avoid an unacceptable worsening of
binary size and load time.
2. Racket very intentionally does not specify the format of zo files,
and indeed the details routinely change: I think there have now
14 such changes on Git since the 8.0 release in February. Continuing
to patch zo files sets us up for future breakage, and it seems like
it would be especially easy for maintainers of the Guix package to
miss the implications of such changes to low-level implementation
details (as I did!). For example, Chez Scheme seems to make
compression options programmer-configurable even within a single
object file: if Racket exposed such options, we could well end up
More broadly, I think the best strategy for Racket packaging will be, as
much as possible, to use Racket's supported configuration features
rather than Guix-specific hacks. This seems especially viable since
Racket has been willing to accept unobtrusive patches upstream that help
things go smoothly for Guix, e.g. with 8.1 we should no longer need any
patches to the build scripts: we're all friendly, parentheses-loving folks.
For another example, it looks like existing
"racket-store-checksum-override.patch" fixes a previous issue discussed
in <https://issues.guix.gnu.org/30680> caused by grafting zo files: I
hope this change will also let us remove that patch, though I'd want to
test more before proposing we drop it. So these problems aren't
fundamentally new; they just have an additional symptom since the change
to Racket CS by default in Racket 8.0. If we can fix the root problem of
violating Racket's assumptions by patching zo files, we should be able
to stop hunting down symptoms.
Rather than using "config.rktd", an alternative approach would be to set
things up so that `dlopen` would find the foreign libraries, perhaps via
`LD_LIBRARY_PATH`. This has some intriguing possibilities: I could
imagine Guix providing an alternate `dlopen` implementation that might
be useful beyond Racket.
Nix apparently configures some things via `LD_LIBRARY_PATH`, but their
approach (as I understand it) relies on generating Racket scripts around
all Racket-generated executable, which causes other problems. There
should be workarounds, but it seems better to avoid going down that road
if we can.
Finally, here's a sketch of how `guix import racket` and such might
work. Racket's package system has a concept of "package scope", so that
"installation" scope can coexist with narrower scopes (mostly per-user
scope, though there are more complex possibilities). Right now, Guix
puts installation scope in the read-only store, which basically
corresponds to how other package managers put it in root-owned places,
except Guix can't write to the store to install additional packages. I'm
still at the information-gathering stage, but my current thinking is
that the hypothetical `racket-build-system` should basically take the
source package and turn it into what Racket calls a "built" package
ready to be installed in `static-link` mode, which includes compiling
the code and building the docs (which can involve quite a lot, e.g. ray
tracing icons). Then a profile hook could knit together all of the
Racket packages into an installation package scope. For packages that
depend on foreign libraries, this would be a chance for Guix to add the
necessary paths to the "config.rktd" for the installation.