[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#32190: Cuirass doesn't check if two subsequent jobs yield the same d
From: |
Ludovic Courtès |
Subject: |
bug#32190: Cuirass doesn't check if two subsequent jobs yield the same derivation |
Date: |
Tue, 24 Jul 2018 12:05:35 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) |
Hi Clément,
Clément Lassieur <address@hidden> skribis:
> Consider the following table:
>
> CREATE TABLE Derivations (
> derivation TEXT NOT NULL,
> evaluation INTEGER NOT NULL,
> job_name TEXT NOT NULL,
> system TEXT NOT NULL,
> nix_name TEXT NOT NULL,
> PRIMARY KEY (derivation, evaluation),
> FOREIGN KEY (evaluation) REFERENCES Evaluations (id)
> );
>
>
> And the following code:
>
> (define (db-add-derivation db job)
> "Store a derivation result in database DB and return its ID."
> (catch 'sqlite-error
> (lambda ()
> (sqlite-exec db "\
> INSERT INTO Derivations (derivation, job_name, system, nix_name, evaluation)\
> VALUES ("
> (assq-ref job #:derivation) ", "
> (assq-ref job #:job-name) ", "
> (assq-ref job #:system) ", "
> (assq-ref job #:nix-name) ", "
> (assq-ref job #:eval-id) ");")
> (last-insert-rowid db))
> (lambda (key who code message . rest)
> ;; If we get a unique-constraint-failed error, that means we have
> ;; already inserted the same (derivation,eval-id) tuple. That happens
> ;; when several jobs produce the same derivation, and we can ignore it.
> (if (= code SQLITE_CONSTRAINT_PRIMARYKEY)
> (sqlite-exec db "SELECT * FROM Derivations WHERE derivation="
> (assq-ref job #:derivation) ";")
> (apply throw key who code rest)))))
>
> I think the above constraint can't happen because by definition a new
> job (for the same job_name) is produced at each evaluation. So eval-id
> will be incremented every time.
I added it at the time because it did happen. In a given eval, there
can be two jobs producing the same derivation (for instance a job called
“gcc” produces xyz-gcc-5.5.0.drv, and a job called “gcc-5.5.0” produces
the very same xyz-gcc-5.5.0.drv.)
> Also, the docs (and a comment in schema.sql) says:
>
> Builds are not in a one to one relationship with derivations in
> order to keep track of non deterministic compilations.
>
> But I think it doesn't make sense, because Guix won't try to build twice
> the same thing unless '--check' is used (which obviously isn't the
> case).
The rationale (that was back in Mathieu’s GSoC) was that sometimes, you
can have several builds logs for one derivation. In Hydra this happens
if a build fails for some non-deterministic reason and then you click on
“Restart” in the hope that it’ll succeed this time. ;-) In this
situation Hydra keeps both build logs IIRC.
Anyway, I lean towards keeping only one build log, at least for now,
which is what guix-daemon does.
> So not only we have a huge Derivations table full of identical items,
> but we also ask Guix to build them and we store the results in the
> Builds table...
>
> Maybe the solution is to replace the (derivation, evaluation) primary
> key with (derivation), and only build the newly added derivations.
> WDYT?
I agree, we don’t need all these identical items, it makes no sense.
You can go ahead and clean that up! ;-)
Thank you,
Ludo’.