[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 1/2] Support publishing build events

From: Christopher Baines
Subject: Re: [PATCH 1/2] Support publishing build events
Date: Tue, 03 Dec 2019 20:44:24 +0100
User-agent: mu4e 1.2.0; emacs 26.3

Clément Lassieur <address@hidden> writes:

> Christopher Baines <address@hidden> writes:
>> I guess so… I was probably just thinking about JSON from JSON fields
>> in PostgreSQL, but we're not using PostgreSQL here.
>> The one advantage I can think of is that not every Scheme object can be
>> converted to JSON (at least with guile-json), so by storing the JSON
>> representation, you're guaranteed to have a valid value. If you stored a
>> scheme object, there's a chance that you could store something that was
>> invalid, or at least didn't convert to JSON.
> Do you know any Scheme object that cannot be converted to JSON?  Even
> procedures can.

I was thinking of something like this:

scheme@(guile-user)> (scm->json-string '(()))
json/builder.scm:91:21: In procedure build-object-pair:
In procedure car: Wrong type argument in position 1 (expecting pair): ()

>>> But in that case, why not fetch them from the Guix Data Service?
>> I'm still not sure what the ideal scope and architecture for the Guix
>> Data Service, or Cuirass is, but at the moment I've been trying to avoid
>> making one dependent on the other.
> Well, if we want to keep our softwares (GDS and Cuirass) as simple as
> they can be, there should be a clear delimitation about what each of
> them does, and they should not implement the same features, in my
> opinion.

At least in my mind, this is an important issue, but something I'm very
uncertian about. The biggest existing overlap in my mind is that the
Guix Data Service and Cuirass compute and store derivations.

> Also, I can't see how GDS wouldn't be dependent on Cuirass.

Maybe dependent wasn't the best way to phrase it. What I've been
thinking about is you should be able to run Cuirass without the Guix
Data Service, and this might be very applicable for small deployments.

> So maybe I'm wrong, but if GDS stores the events, I think it's a bad
> idea that Cuirass stores them too.


>> Also, I've been thinking about WebSub when trying to provide a way to
>> subscribe to events, which I'll say more about shortly…
> I'd love to understand what WebSub is too!

So I meant to say more in my last email, but I obviously forgot to

So WebSub is W3C recommendation standard (that doesn't carry much weight
with me, but I was looking for some kind of standard or interoperable
way of doing things). In it's model, I've been thinking of Cuirass as a
"Publisher" [1].


WebSub describes subscripting to topics, through their URL. Which is why
I created URLs to serve the build and evaluation events. I also think
some people may want to just fetch the events, rather than subscribing
to them.

>>> Now that I think about it, I don't understand the point the HTTP API
>>> ("build-events", "evaluation-events").  If events are sent from Cuirass
>>> to the Guix Data Service, why would anyone need to fetch those events
>>> from Cuirass?  Wouldn't it make more sense to fetch them from the Guix
>>> Data Service?
>> So, while this work is trying to copy data from Cuirass to the Guix Data
>> Service, I'm still thinking about them separately. I guess if you're
>> running your own instance of Cuirass, you might want to fetch events
>> from it, without having to setup an instance of the Guix Data Service.

So adding a bit to what I say here, I added the API endpoints also to
keep somewhat aligned with WebSub.

> Is there any use of having raw Cuirass events without the Guix Data
> Service to interpret them?

I think so, you could hook them up to an IRC bot for example, to post in
some channel when events happen to builds.

It's true that the information in the events table can also be found
elsewhere in the database, it's just not in an ideal form to send out to
some receiver.

I'm still interested in getting Cuirass to publish events to a WebSub
hub, at which point a topic URL for build events would be necessary (at
least in terms of WebSub) to subscribe to events.

However, at the moment, I'm trying to do something minimal to get data
from Cuirass in to the Guix Data Service, and I haven't set up the hub
needed for WebSub.

Maybe it would be simpler to remove the API endpoints for now, and they
can possibly come back once there's a WebSub hub involved.

>>> Why do we need to use a fiber, if there is only one?  A fiber is a
>>> lightweight thread.  They are useful if we need to do several things in
>>> the same time, but if there is only one thing to do, I don't understand
>>> the point.
>> If I remember correctly, the way Cuirass accesses the database is tied
>> up with fibers, so it's to make the database access work here.
> It's not tied up with fibers.  There is a database server in another
> thread, and WITH-DB-CRITICAL-SECTION will allow you to communicate with
> that server through channels.  But you don't need to be in a fiber to
> communicate through channels, being in the main thread is good enough.

Ok, I'll have another look at how fibers are used when sending out



Attachment: signature.asc
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]