Re: Guix pull: avoiding "Computing Guix derivation"

[Richard Sent]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On lun., 13 mai 2024 at 17:11, Richard Sent <richard@freakingpenguin.com> 
> wrote:
>
>> Instead of A and B building C directly, A and B download the
>> substitutable Guix package D, then use D to build C. Because D is a
>> reproducible package, it should be substitutable from both A and B.
>> Then, because D->C is the same for everyone, we could substitute the
>> Guix derivation computation for C.
>
> Maybe I am missing some details.  From my understanding, your D is the
> result of the “Computing derivation” dance.  And it is a minimalist
> build because it must work for both cases: with or without substitutes.
>
> Somehow, my understanding of the current process is:
>
>   A -> D  -> C
>   B -> D* -> C
>
> And, D or D* can be the same script.  Here, the property is a kind of
> idempotency.  Hence, C is substitutable.
>
> IIUC, your proposal is to fix D (the intermediary step).  You propose
> the package named ’guix’ that changes barely, but it could be something
> more minimalist.  The requirements is: susbtitutable.  The problem is
> transferred to the first arrow, no?
>
> How can we be sure that A and B points to the same D?
>
> Other said, A lives in the past.  Later, we changed D becoming D*
> because some bugs.  The other revision B lives after this change.
> Therefore, we have the same picture:
>
>   A -> D
>   B -> D*
>
> But then how do we go to C since D and D* does not have a kind of
> idempotent property.
>
> From my understanding, the current situation/process is very similar
> with the one for compiling compiler: you have your new Source of
> compiler and an OldCompiler binary, so you run:
>
>   OldCompiler(Source) -> Compiler
>   and again Compiler(Source) -> NewCompiler
>
> Somehow, whatever the choice of some OldCompiler binary, you get the
> same NewCompiler binary (aside trusting trust attack and friends,
> obviously ;-))

You're correct. This solution wouldn't be sufficient to avoid "Computing
Guix Derivation" for every possible A or B. To my understanding it could
reduce the frequency this occurs.

(To change letters yet again, I'll use Z to refer to the output Guix.)

As a package (either "guix" or a minimalist subset), D, D*, D**, etc.
should all be substitutable. If we consider input Guix's A, B, C, E, F,
..., our original "Compute Guix Derivation" graph would look like:

--8<---------------cut here---------------start------------->8---
     Z
    / \
   /| |\
 /| | | |\
/ | | | | \
A B C E F G
--8<---------------cut here---------------end--------------->8---

i.e. for every possible input Guix (and output Z), we'd need to store a
substitutable intermediate step. Not feasible.

By having A, B, C, ..., use a substitutable Guix package D, our graph
would look like this instead:

--8<---------------cut here---------------start------------->8---
      Z
     /| \
    / |  \
  D   D*  D**
 /|\  |\  |
A B C E F G
--8<---------------cut here---------------end--------------->8---

Assuming D changes significantly less frequently than A, B, C..., I
would think this should be something we could feasibly substitute (at
least for recent D and Z).

Perhaps this isn't viable. I'll confess to not being super familiar with
how build-self.scm operates. At a naive level, $ guix build guix doesn't
need to run "Computing Guix derivation", so if A->D doesn't need to
compute the derivation, and D->Z is substitutable for some subset of D
and Z, I'd think we can skip "Computing Guix derivation" as long as
someone isn't trying to either

a) Use an A that refers to a D********** that's since been GCd

b) Try to get a Z that hasn't been built by D, D*, D**, etc.

Does that sound correct?

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.