Re: system/package testing?

[Maxime Devos]

On 06-10-2022 17:51, Simon Josefsson via Development of GNU Guix and the 
GNU System distribution. wrote:
> Hi
>
> Is there any infrastructure to run a system-wide test suite, or at least
> a per-package test suite?

There's "make check-system", for system services (and implicitly, for 
the software behind those services), and "make check", for individual 
packages.

>  I'm not thinking of 'make check' that is run
> during build, but of something similar to Debian's debci/autopkgtest to
> test that an installed package works as expected (which may include
> running the same self-checks as 'make check' would, but on the installed
> package).

I do not understand the difference -- what's installed, is just what's 
built previously, on which "make check" is run, no?

If there are some additional tests beyond "make check", they can be run 
from the check phase (and if they require the package to be installed, 
the phase can be moved after 'install', as is done sometimes already for 
some Python packages).

> When designed for Guix, it could be even better than debci/autopkgtest,
> it could be something the Guix user could invoke on an installed system
> to verify that all the installed packages passes self-tests

This is done automatically, during the build.  If there is a concern it 
depends on the kernel version or CPU model, "guix build --check" can be 
used to redo the build.

 -- I miss
> that functionality from debci/autopkgtest.
>
> I'm considering packaging libgssglue for Guix and do some Kerberos
> related work on other related packages, but having confidence of any
> changes without proper testing of the resulting installed package is
> fragile.

If you have additional tests for libgssglue, you can submit them 
upstream, or if they are somehow inappropriate for upstream, do them in 
a check phase.

> I'm sure other more complex parts of Guix could benefit from
> similar testing too.  A system-wide testing of the published QEMU image
> would be useful, to gain confidence that we don't introduce regressions
> into a future release of the QEMU image.

See: "make check-system".

> Maybe this already exists, and if so my question/request should be
> converted into a request for publicity around the feature and a pointer
> to its documentation. :)

Info: (guix)Running the Test Suite

Greetings,
Maxime

OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key

OpenPGP_signature
Description: OpenPGP digital signature