List of records in G-Exp

[Martin Baulig]

Hello,

I'm currently working on a somewhat complex service; there are multiple 
daemons, each running as its own user, and some "secrets" that need to be 
installed (passwords, X.509 certificate keys). After getting a prototype up and 
running a couple of weeks ago, I'm now trying to polish things up and remove 
some of the ugly hacks that I had used in my first implementation.

To make this modular and extensible, there will be several services that build 
on top of each other:

- The service-account-service-type at the bottom, extending both 
account-service-type and activation-service-type to create the user accounts 
and their data directories (each daemon will have its own log file, pid file 
and scratch storage, etc.)
- The secrets-service-type extends that to install some files that only the 
daemons can access.
- The actual services sit on top of that - and extend the first two based on 
their configuration.

I got a prototype working, but my code looks rather ugly, so I was wondering 
whether there might be a better way.

I defined these two record types:

> (define-record-type* <service-account-service>
> service-account-service
> make-service-account-service
> service-account-service?
>
> (root service-account-service-root (default "/etc/private"))
> (entries service-account-service-entries (default '())))
>
> (define-record-type* <service-account>
> service-account
> make-service-account
> service-account?
>
> (id service-account-id)
> (user service-account-user (default "root"))
> (group service-account-group (default "root"))
> (create-account? service-account-create-account (default #f))
> (working-directory service-account-working-directory (default #f))
> (has-daemon? service-account-has-daemon? (default #t))
> (pid-directory service-account-pid-directory (default #f))
> (log-directory service-account-log-directory (default #f))
> (log-file service-account-log-file (default #f)))

And then I declare my service type:

> (define-public service-account-service-type
> (service-type (name 'service-account)
> (extensions
> (list (service-extension activation-service-type
> service-account-service-activation)))
> (compose concatenate)
> (extend extend-service-account-entries)
> (description
> "Service-Account service activation.")
> (default-value (service-account-service))))

and I can then extend it via something like

> (define secrets-service-account
> (match-lambda
> (($ <secrets-service> _ _ entries)
> (map secrets-entry-account entries))))
>
> (define-public secrets-service-type
> (service-type (name 'secrets)
> (extensions
> (list (service-extension activation-service-type
> secrets-service-activation)
> (service-extension service-account-service-type
> secrets-service-account)))
> (compose concatenate)
> (extend (lambda (config extended-secrets)
> (match-record config <secrets-service>
> (database root-directory entries)
> (secrets-service
> (database database)
> (root-directory root-directory)
> (entries (append entries extended-secrets))))))
> (description
> "Secrets service activation.")
> (default-value (secrets-service))))

This is all pretty much unspectacular.

The problem arises with the activation service implementation:

> (extensions
> (list (service-extension activation-service-type
> service-account-service-activation)))

If I understand this correctly, then my service-account-service-activation 
needs to be a function that takes an instance of my <service-account-service> 
record as its only parameter and returns a G-Exp.

So ...

> (define (service-account-service-activation configuration)
>
> #~(begin
>
> (format #t "Activating service accounts: ~a~%" #$configuration)))

Well, to - that's going to produce an error complaining about my 
<service-account-service> record not being a valid G-Exp input.

I can access its two string fields easily - but how do I iterate over the list 
of <service-account> records? I'd have to use ungexp inside that inner block - 
but that only seems to work with stuff that can be "lowered".

Of course, I could "cheat" and do something like ...

> #$@(map some-func (service-account-service-entries configuration))

... and that both compiles and runs (provided that I put that some-func into an 
imported module) - but that some-func will be invoked multiple times throughout 
a "guix system reconfigure".

But I want to do it the right way.

Here's what I came up with:

First, we define a function that turns a <service-account> into a list of 
primitive types (strings, integers, booleans):

> (define translate-account
> (match-lambda
> (($ <service-account> id user group create-account? working-dir
> has-daemon? pid-dir log-dir log-file)
> `(list ',id ,user ,group ,create-account? ,working-dir
>
> ,pid-dir ,log-dir ,log-file))))

Then, I define a G-Exp compiler for <service-account-service>:

> (define-gexp-compiler (service-account-service-compiler
> (config <service-account-service>)
> system target)
> (with-monad %store-monad
> (match config
> (($ <service-account-service> root entries)
> (return `(list ,root
> ,@(map (lambda (account)
> (translate-account
> (resolve-service-account root (cdr account))))
>
> entries)))))))

And finally, in my service-account-service-activation, I pattern-match against 
these lists:

> (define (service-account-service-activation configuration)
> (with-imported-modules '((baulig build utils))
> #~(begin
> (use-modules (baulig build utils))
> (match-let (((root . entries) #$configuration))
> (format #t "Activating service accounts: ~a~%" root)
> (map (match-lambda
> ((id user group create-account? working-dir pid-dir log-dir log-file)
> (format #t "Activating accound: ~a - ~a / ~a - ~a - ~a - ~a - ~a / ~a~%"
> id user group create-account? working-dir pid-dir log-dir log-file)))
>
> entries)))))

Well, it compiles and runs - but it's still quite a bit of boilerplate.

And ideally, I'd like to have a function that takes an instance of a 
<service-account> record and returns a G-Exp containing all the mkdir-p, chown, 
etc. commands for it. These cannot return a "normal" derivation / store item - 
because their outputs need to be installed with special ownership / file 
permissions, as they will contain passwords / keys required by the daemons to 
run.

But since there will be multiple instances of that record, I'd get some 
function returning a list of G-Exp's. To sequentially execute them, my gut 
feeling tells me that I'll need to use the monadic nature of the Store - is 
there something like Haskell's Monadic map functions for G-Exp's?

Full code:

- 
[account.scm](https://gitlab.com/martin-baulig/config-and-setup/guix-packages/-/blob/662a8142562942306bb4746a8058e13109dee1a8/packages/baulig/services/account.scm)
- 
[secrets.scm](https://gitlab.com/martin-baulig/config-and-setup/guix-packages/-/blob/662a8142562942306bb4746a8058e13109dee1a8/packages/baulig/services/secrets.scm)

And I'm trying to replace [this 
monstrosity](https://gitlab.com/martin-baulig/config-and-setup/guix-packages/-/blob/662a8142562942306bb4746a8058e13109dee1a8/packages/baulig/services/bacula.scm).

I'm still fairly new to GNU Guix, so any feedback would be greatly appreciated.

Best regards,

Martin