[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [GSoC] Development of Cuirass.
Re: [GSoC] Development of Cuirass.
Mon, 20 Mar 2017 23:43:46 +0100
Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
address@hidden (Ludovic Courtès) writes:
>> 2.1 The testing infrastructure
>> Currently Cuirass is providing a small test suite consisting only of
>> unit tests. The integration tests are done manually with basic
>> examples consisting in building small packages such as GNU Hello.
>> This is not good, because actual packages take a consequent time to
>> compile, and testing only basic examples leaves the interesting
>> complex case out of the scope of testing. The worst is that the
>> result depend on the phase of the moon since every commit in Guix can
>> potentially break the build.
>> The idea is to take inspiration from the Guix test suite by providing
>> tiny mocked packages that will help having lightweight build
>> specifications. Those tests require launching another instance of the
>> `guix-daemon' with a local temporary store.
> Sounds like a good idea.
> It would be nice to refine what it is we’re trying to test and that’s
> not addressed by unit tests. At one end of the spectrum, I imagine we
> could be spawning a couple of GuixSD VMs connected together, one serving
> a Git repo, and the other one running Cuirass pulling from that repo (a
> system test).
> At the middle of the spectrum is something like you describe, I think:
> run Cuirass directly upon “make check”, though there are complications:
> what repo do we pull from, do we really want to build packages and if
> not how to we mock builds, etc.
The idea I had was to create fake vcs repositories programmatically. I
imagine that it is possible to associate some dummy build script that we
can deterministically make fail.
> There are several components we’d like to test more closely IMO: SCM
> polling, evaluation, and job scheduling. Perhaps some of that can also
> be achieved through unit tests?
Yeah, that would be better.
>> 2.2 HTTP API
>> Since commit [12d71ee098c3eb84a9a5dc2419bd37a3b9e55dd2] Cuirass has a
>> web server which is running in a separate thread of the `cuirass'
>> process. However the resources it provides are pretty limited since
>> you can only query the lists of specifications currently watched.
>> This is done with the `/specifications' or `/jobsets' resource names.
>> "Jobsets" is the terminology used by Hydra and is used here for
>> compatibility with its HTTP API.
> For a start, I think we should implement all or a subset of the Hydra
> API, as-is (it’s currently used by Emacs-Guix):
> A large part of it is about monitoring the status of Hydra: queued
> builds, recently-completed builds, jobs, etc. That is the most pressing
> need, I think.
>> This roadmap lacks details about the concrete tasks and its associated
>> timeline. However I prefer having some feedback about this proposal
>> first, before providing a complete roadmap.
>> • From May 30 to June 26, I will work on the testing infrastructure,.
>> • From June 27 to August 29, I will work on the HTTP API.
> You may find that you’ll want to interleave tasks. I don’t know about
> you, but sometimes I get bored if I keep working on the same task or
> domain for too long, so switching to something else for a while can be a
> relief. :-)
I don't know, I have been rarely confronted to that issue personnaly
(more the opposite: too much context switching). But I will
preventively adapt my roadmap.
> There are a couple of isolated tasks that come to mind, which do not
> really fit in the proposal but are worth looking into as time permits:
> • improved error reporting upon evaluation failures and so on;
> • improved error recovery;
I totally forgot to precise that the error handling issue is exactly the
reason I think we need a better test infrastructure for Cuirass. Being
able to reproduce various errors or failues allows a better confidence
in the error being properly handled. I find it hard to just imagine
and write the appropriate handler.
> • use of Guile-Git instead of Git.
I Will add this to the plan.
GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37
- [GSoC] Development of Cuirass., Mathieu Lirzin, 2017/03/12
- Re: [GSoC] Development of Cuirass., Ludovic Courtès, 2017/03/13
- Re: [GSoC] Development of Cuirass.,
Mathieu Lirzin <=
- Re: [GSoC] Development of Cuirass., Andy Wingo, 2017/03/13
- Re: [GSoC] Development of Cuirass., Efraim Flashner, 2017/03/13
- Re: [GSoC] Development of Cuirass., Mathieu Lirzin, 2017/03/21
- Re: [GSoC] Development of Cuirass., Ludovic Courtès, 2017/03/21
- Re: [GSoC] Development of Cuirass., Jan Nieuwenhuizen, 2017/03/21
- Re: [GSoC] Development of Cuirass., Tobias Geerinckx-Rice, 2017/03/21
- Re: [GSoC] Development of Cuirass., Leo Famulari, 2017/03/21
- Re: [GSoC] Development of Cuirass., Ricardo Wurmus, 2017/03/22