Re: How did you handle making a GNU/Linux distribution?

[Sage Gerard]

Thank you for the links!

> I miss which problem Xiden is solving and how it does.
>

You are not the first to say so, and I'm happy to learn more about why.

I'll try to explain in a different way here, so forgive the text wall. I'll incorporate any feedback here to improve the white paper. I'd also like to hear more about Guix's philosophy on what I'm going to talk about, because I had reasons to end up with a different model.

--

    If we each use a program to distribute software, then those programs
    are probably incompatible in some way (e.g. Cargo vs. Nexus'
    Vortex). Xiden addresses this by decoupling the subjective elements
    of software distribution from the objective ones.  That means
    modeling software distribution as (partly) a semantics problem.
    Racket didn't do this with its own package managers. Which is
    ironic, because it's hard to guarantee that that an
    arbitrary Racket program will get the dependencies they mean
    to use. I've written about this at length [1][2][3], and spoke about
    it last year [4]. Xiden has changed a bit since the speech to be
    more flexible, and it is no longer limited for use in Racket
    projects. Apologies if you see something that looks contradictory.

What does that mean? Let's say Alice and Bob each write a Xiden launcher.

Alice

• trusts SHA-384, but only if its implementation uses Keccak.
• defines "sha384" as the canonical name of the CHF, and treat variants like "SHA-384" as aliases.
• delegates trust in artifact signatures to GPG subprocesses.
• downloads content from her employer's S3 buckets.
  
• unconditionally prefers cached installations, such that every defined artifact has exactly one stored copy on disk. All dependents access the stored artifact using symbolic links or Windows junctions.
  
• prefers declaring versions with edition and revision names.

Bob

• trusts SHA-1, but only for artifacts from services authenticated by his operating system's certificates.
• trusts digest creation and signature verification as implemented by his host's dynamically--linked `libcrypto` library for use in the same process.
• downloads content from GitHub packages
  
• unconditionally prefers SxS installations, such that packages cannot conflict in a shared namespace at the cost of defeating an internal cache.
  
• prefers Semantic Versions

These preferences are valid in Xiden's DSL for writing launchers. The invariants for each launcher can differ as much as Cargo and Vortex differ. What I wanted to do was allow users to declare their own invariants explicitly, but only when those invariants are wanted for subjective reasons. I can't just write an overly-opinionated tool in this space because opinions have shelf-lives, and they can't interrupt our ability to get the content we need on demand, with our own safety guarentees.

Xiden launchers have advantages over shell scripts in that we can both still download and install dependencies from the same servers, and create reproducible builds in terms of the same (bit-identical) state on disk. We just differ on details that shouldn't impact how we work. It also helps us patch holes faster, since if (say) a catalog maintainer tells us that a CHF was compromised, we can immediately revoke trust in the CHF independently in our own clients. The package definitions are all expressed in terms of what the launchers allow.

So what problem does this solve? I'm trying to preserve an element of walk-away power in any tech community. Maybe your community goes in a questionable direction. Maybe your desired software is in a different ecosystem. Maybe the maintainers are abusive jerks. Maybe you have a `leftpad`/`event-stream` incident and the developer is unable/unwilling to patch the problem. You'd probably want a way to escape to a new community or distribution model without giving up content or doing complex integration work. If Xiden can do that, users don't have to depend on the same middlemen to share work. This is also better for developers because it doesn't oblige them to do everything their users want, while knowing users will adapt to inevitable distribution problems. Everyone's consent becomes explicit, such that Xiden's "real" invariant is that consent between any two parties be mutually compatible. I hope to improve user freedom from this angle, and my approach is biased by my experience with Racket's limitations in this space.

The subjective parts I talk about go further into details like what name canons, versioning schemes, trusted certificate chains, and even specific approaches to an exact diamond dependency. The objective parts include integrity checking, signature verification, safety limits, automatic dependency resolution, patches, protocols, and other concerns that a decent package manager normally cares about. By keeping these separate, all of the below features become cheap to write, without sacrificing trust & safety in the abstract.

• Download archived GitHub repositories (source on Racket Discord atm)
• Manage Racket installations like how `nvm` manages Node.js installations [5]
  
• Produce a self-hosted Xiden installations, with implicit trust for the running Xiden instance [6]
• Control which exact artifacts must be produced deterministically [7]
• Force generated bindings to be `eq?` (This one haunts Racket's package managers today) [8]
  
• Source package definitions from Pastebin to download content, digests, and signatures from a public S3 bucket (This does not exist today. I'm spitballing because I know it would work)

I hypothesize that Xiden can "abstract over" package managers in the same way that Racket can abstract over languages. I think people are used to package managers deciding things for them.

So what is Xiden for? Well, it's not for deciding an SOP for you. It's for giving everyone the option to change the SOP for themselves with minimal politics.

> what you would like to bootstrap?

All binaries related to creating a GNU/Linux distribution, such that I can reproduce an exact OS, Racket installation, and Xiden instance. I want a trusted initial state on GNU/Linux.

To be clear, I don't need to do this for functional reasons. Xiden's already operational. I just can't claim that you can trust a given Xiden instance with the same confidence as a Guix instance right now.

Caveats:

• I have no idea how to do this yet, so thanks to all of you for the leads.
  
• Xiden is cross-platform, and it already runs anywhere Racket can (Windows, macOS, Unix-likes). I can't make the best claims re: trustworthiness without GNU software, but Xiden is at least operational everywhere. That's why I ended up defining a small attack surface that's consistent across systems, to make it easy to set up host-level protections.

> If you mean a trusted seed in order to start a package collection, you could use the current Guix binaires---as a starting point.
>

I'm reading GNU Mes' manual. I am confident that it will help me find the answers I'm looking for, including all of your helpful replies!

[1]: https://sagegerard.com/polyglot3-package-nightmare.html
[2]: https://www.reddit.com/r/GUIX/comments/p8v5xd/how_did_you_handle_making_a_gnulinux_distribution/h9zpyhn?utm_source=share&utm_medium=web2x&context=3
[3]: https://sagegerard.com/new-racket-pkg-releases.html
[4]: https://www.youtube.com/watch?t=2330&v=bIi-tUzOwdw&feature=youtu.be
[5]: https://github.com/zyrolasting/xiden/tree/master/examples/racket-installation-manager
[6]: https://github.com/zyrolasting/xiden/tree/master/examples/self-hosting
[7]: https://github.com/zyrolasting/xiden/tree/master/examples/determinism
[8]: https://github.com/zyrolasting/xiden/tree/master/examples/generated-racket-bindings