Using cuirass to build your own manifest.

[Mathieu Othacehe]

Hi,

Here's a small tutorial on how to setup cuirass to build your own
manifest.

I see two major reasons for this kind of setup:

* When you pull latest guix, hydra and bayfront may not have finished
  building all the packages you use.

* Hydra and bayfront won't build your custom packages.

For this reasons I installed cuirass to build my manifest, using a
configuration very similar to the one used on bayfront
(http://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/bayfront.scm).

I guess it may not be the easier way to do it so don't hesiste to
comment. Anyway, here's how I set it up.

1) config.scm

--8<---------------cut here---------------start------------->8---
(define %cuirass-specs
  ;; Cuirass specifications to build Guix.
  #~(list `((#:name . "guix")
            (#:url . "git://git.savannah.gnu.org/guix.git")
            (#:branch . "master")
            (#:no-compile? #t)
            (#:load-path . ".")
            ;; Eval function drv-list in file guix-drv.scm
            (#:proc . drv-list)
            (#:file . #$(local-file 
"/home/mathieu/conf/guix/cuirass/guix-drv.scm")))))
(services
 ...
 (service cuirass-service-type
          (cuirass-configuration
           (interval 30) ;; git pull guix repo every 30 seconds.
           (use-substitutes? #t)
           (port 8082)
           (load-path '("/home/mathieu/conf/guix/packages"
                        "/home/mathieu/conf/guix/common_packages"))
           (specifications %cuirass-specs))))
--8<---------------cut here---------------end--------------->8---

With this configuration, cuirass service will wake-up every 30
seconds. It will pull branch master of guix.git. If new commits have
appeared since last evaluation, a build will be triggered.

Cuirass will evaluate the function specified with #:proc in the file
#:file. This function is supposed to return the list of derivations you
want to build.

Note that if this list contains custom packages, you have to setup
load-path to point to the directories containing the custom packages
definitions.

2) guix-drv.scm

--8<---------------cut here---------------start------------->8---
(use-modules (guix config)
             (guix store)
             (guix grafts)
             (guix packages)
             (guix ui)
             (guix derivations)
             (guix monads)
             (guix profiles)
             (gnu packages)
             (srfi srfi-1))

(define (drv-package store package)
  (lambda ()
    `((#:job-name . ,(string-append
                      (package-name package)
                      "-"
                      (package-version package)
                      "-job"))
      (#:derivation . ,(derivation-file-name
                        (parameterize ((%graft? #f))
                          (package-derivation store package #:graft? #f)))))))

(define (drv-list store arguments)
  (let* ((manifest
         (load* "/home/mathieu/conf/guix/manifest.scm"
                (make-user-module
                 '((guix profiles) (gnu)))))
         (packages
          (map manifest-entry-item
               (manifest-entries manifest))))
    (parameterize ((%graft? #f))
      (map (lambda (package)
             (drv-package store package))
           (delete-duplicates! packages)))))
--8<---------------cut here---------------end--------------->8---

The drv-list procedure loads the file manifest.scm which content is
detailed below. The list produced by drv-list looks like :

--8<---------------cut here---------------start------------->8---
(((#:job-name . "acpi-1.7-job") (#:derivation
. "/gnu/store/r9s5x0ksj02hsw4n3acdxab8ggjp4z7y-acpi-1.7.drv")) ...)
--8<---------------cut here---------------end--------------->8---

3) manifest.scm

--8<---------------cut here---------------start------------->8---
(define (spec->packages spec)
  (call-with-values (lambda ()
                      (specification->package+output spec)) list))

(define packages-list
  '("acpi"
    "acpica"
    ...
    "yasm"
    "zip"
  ))

(packages->manifest
 (map spec->packages packages-list))
--8<---------------cut here---------------end--------------->8---

Now you may want to run cuirass on a dedicated machine and access
packages built by cuirass on other machines. You just need to run guix
publish to do that.

4) config.scm

--8<---------------cut here---------------start------------->8---
(services
    ...
    (guix-publish-service #:host "0.0.0.0"))
--8<---------------cut here---------------end--------------->8---

5) other machines config.scm

--8<---------------cut here---------------start------------->8---
(services
 (modify-services %base-services
                  (guix-service-type
                   config =>
                   (guix-configuration
                    (inherit config)
                    (substitute-urls '("https://bayfront.guixsd.org";
                                       "https://mirror.hydra.gnu.org";
                                       "https://cuirass-machine-name";))))))
--8<---------------cut here---------------end--------------->8---

Where "cuirass-machine-name" is the URL of the machine running cuirass.

Note that the order in substitute list has an importance. Here, the
substitutes from cuirass-machine-name are only downloaded if they do not
exist on bayfront and hydra.

Depending on the upload speed of your build machine you may want to put
it on top of the list or not ...

At last, you need to authorize "cuirass-machine-name" key with this
command :

--8<---------------cut here---------------start------------->8---
guix archive --authorize < cuirass-machine-name.pub
--8<---------------cut here---------------end--------------->8---

You can refer to
https://www.gnu.org/software/guix/manual/html_node/Substitutes.html#Substitutes
for precisions.

That's it, I hope it will be helful.

To finish, the downsides of this setup are :
* You need to keep your manifest up-to-date on your build machine.
* There are no easy ways to track cuirass status but to do some sql on
cuirass database.

Besides that, it's working fine.

Happy building,

Mathieu