[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[bug#47180] [PATCH] gnu: racket: Don't inject store paths into Racket fi

From: Philip McGrath
Subject: [bug#47180] [PATCH] gnu: racket: Don't inject store paths into Racket files.
Date: Sun, 11 Apr 2021 23:40:59 -0400
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.9.0


On 4/10/21 4:59 PM, Ludovic Courtès wrote:
Hi Philip,

Philip McGrath <> skribis:

Apparently, during grafting, Guix can somehow mangle compiled
Racket CS files (.zo) such that Racket will refuse to load them.
(Maybe it has something to do with compression?)

If those files are compressed, and if a store file name survives despite
compression, then grafting can patch it, which could lead to checksum
mismatches or similar.

Yes, that's what seems to be happening.

What error message does Racket produce?

The first error I heard of (and reproduced) was reported by Jack in <>:

$bytevector-uncompress: internal error uncompressing #"\0\0\0\0chez\310\224\206:\r()#\201\256R-d\205\233\24\363\5\20\201P\6A\v\300\0\16\f\6\31\2\f\6\f&H\275\0\1\0\362\bA\377e\0\1\0C\6A\21\3\v\300\0\201\265!\f\6\n\0\a\1\35\0\1+\0\360\27\201\375\300\0\0\0\17\205\210Z\0\0M\215\245\b\4\0\0M9fH\17\206fZ\0\0I\2...
body of "/gnu/store/mmrax3f1vx3c8ih9hhgffpvka6chk96w-racket-8.0/share/racket/pkgs/gui-lib/mred/private/wx/gtk/utils.rkt" body of "/gnu/store/mmrax3f1vx3c8ih9hhgffpvka6chk96w-racket-8.0/share/racket/pkgs/gui-lib/mred/private/wx/platform.rkt"

The error message is referring to an internal Chez Scheme primitive.

The report at <> seems to me to be an alternative manifestation of the same problem: the hexdump there is useful and explains why I did see some of the store path when I had attempted to investigate using `strings`.

So there seem to be at least three bad cases:

 1. The grafter can mangle .zo files so that Racket can't
    read them at all.
 2. The grafter can miss store references in .zo files, so Racket could
    end up using the ungrafted versions.
 3. With a garbage collection, Racket could try to use the ungrafted
    versions but fail to find them at runtime.

So, we stop patching Racket sources with absolute paths to store
files (i.e. for foreign libraries to dlopen).
Instead, we put them in a data file that doesn't get compiled or,
in one case, embed it in C.

That solves the problem for Racket itself, but wouln’t Racket libraries
have the same issue?

Because the problems are triggered by grafting, they only affect packages that have been patched by Guix. For now, Guix doesn't have the ability to install more Racket packages. In the longer term, the one-sentence answer is that it's always possible we might find and switch to a better approach, but this seems most promising. (A bit more of my current thinking on that toward the end.)

Would it be an option to instead turn off compression and keep doing
things as usual?

In theory, this should be possible. I see two significant downsides:

 1. Compiled code would be much larger—maybe twice as big—and, if I
    recall correctly, load times would be worse, too. With the move to
    Racket CS, existing Racket code moved from a world of small and
    cheap bytecode to a world of machine code: the default compression
    settings have been tuned to avoid an unacceptable worsening of
    binary size and load time.
 2. Racket very intentionally does not specify the format of zo files,
and indeed the details routinely change: I think there have now been
    14 such changes on Git since the 8.0 release in February. Continuing
    to patch zo files sets us up for future breakage, and it seems like
    it would be especially easy for maintainers of the Guix package to
    miss the implications of such changes to low-level implementation
    details (as I did!). For example, Chez Scheme seems to make
    compression options programmer-configurable even within a single
    object file: if Racket exposed such options, we could well end up
    here again.

More broadly, I think the best strategy for Racket packaging will be, as much as possible, to use Racket's supported configuration features rather than Guix-specific hacks. This seems especially viable since Racket has been willing to accept unobtrusive patches upstream that help things go smoothly for Guix, e.g. with 8.1 we should no longer need any patches to the build scripts: we're all friendly, parentheses-loving folks.

For another example, it looks like existing "racket-store-checksum-override.patch" fixes a previous issue discussed in <> caused by grafting zo files: I hope this change will also let us remove that patch, though I'd want to test more before proposing we drop it. So these problems aren't fundamentally new; they just have an additional symptom since the change to Racket CS by default in Racket 8.0. If we can fix the root problem of violating Racket's assumptions by patching zo files, we should be able to stop hunting down symptoms.

Rather than using "config.rktd", an alternative approach would be to set things up so that `dlopen` would find the foreign libraries, perhaps via `LD_LIBRARY_PATH`. This has some intriguing possibilities: I could imagine Guix providing an alternate `dlopen` implementation that might be useful beyond Racket.

Nix apparently configures some things via `LD_LIBRARY_PATH`, but their approach (as I understand it) relies on generating Racket scripts around all Racket-generated executable, which causes other problems. There should be workarounds, but it seems better to avoid going down that road if we can.

Finally, here's a sketch of how `guix import racket` and such might work. Racket's package system has a concept of "package scope", so that "installation" scope can coexist with narrower scopes (mostly per-user scope, though there are more complex possibilities). Right now, Guix puts installation scope in the read-only store, which basically corresponds to how other package managers put it in root-owned places, except Guix can't write to the store to install additional packages. I'm still at the information-gathering stage, but my current thinking is that the hypothetical `racket-build-system` should basically take the source package and turn it into what Racket calls a "built" package ready to be installed in `static-link` mode, which includes compiling the code and building the docs (which can involve quite a lot, e.g. ray tracing icons). Then a profile hook could knit together all of the Racket packages into an installation package scope. For packages that depend on foreign libraries, this would be a chance for Guix to add the necessary paths to the "config.rktd" for the installation.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]