Re: [PATCH] thread-safe handling of asynchronous events

[Peter Bex]

On Wed, Jul 05, 2023 at 03:28:54PM +0200, felix.winkelmann@bevuta.com wrote:
> The first patch provides the event-queue mechanism and cleans up the
> scheduler a bit (hiding internal variables while also exposing ##sys#...
> procedures to access them). This also defines hooks that a threading
> API should override to allow suspension/resumption on events.

I'm not super happy with exposing ##sys#fd-list and ##sys#timeout-list as
a procedure under the exact same name that originally held a list.
If there's any code that uses it, that would lead to strange errors.
Might be better to either not expose it at all, or use a completely
different name.

Perhaps now is a good time to make the scheduler into a proper module
that exposes procedures in an unprefixed way?

> The second patch provides a new signal API: "make-signal-handler",
> which creates a handler for one or more signals and "signal-ignore" and
> "signal-default" (to ignore a signal or set the default disposition).
> This is roughly modelled after the Racket API. The old "signal-handler"
> and "set-signal-handler!" procedures have been deprecated.
> 
> The third patch provides "make-finalizer", inspired by Chez' guardians,
> but slightly different to allow blocking/non-blocking tests for
> finalizations. The old API is still available, as often finalization
> performs only very basic operations independent of the currently
> executing context. Finalization procedures can be composed, as a
> finalizer itself becomes subject to finalization once all associated
> objects have been collected. Building guardians on top of this
> should be straightforward, but the latter does not (to my knowledge)
> allow blocking a thread until a finalizer triggers so I chose not
> to implement this interface.

This API looks nice, but as I understand it, make-finalizer requires
all objects in each "pool" to be known when calling it.  AFAIK this
means that when you are creating objects on-the-fly, you'd have to
call make-finalizer N times, and when retrieving objects you'd have
to poll N finalizer "pools", and therefore you can't block on all
objects you'd want to collect.

So what's missing is a way to add an object to an existing finalizer.
Note that this is what guardians support too - when you call a
guardian procedure with an argument, that object is added to the pool.

A (somewhat hacky) way to do this would be to have the finalizer closure
check whether the "mode" argument is a boolean.  If "mode" is not a
boolean, it is assumed to be an object to add to the queue.
Perhaps with an extra check that it's not an immediate value.

Alternatively, we could steal (part of) MIT Scheme's (undocumented)
finalizer API:

https://git.savannah.gnu.org/cgit/mit-scheme.git/tree/src/runtime/gcfinal.scm?id=176cd871bdd9c9dabcb8ad602da0b618be2d0373#n36

There, you call (make-gc-finalizer) with the finalization procedure,
a predicate for the type of object that may be collected, a procedure
that can be used to extract the "context" from the collectable object
and a procedure that can be used to set the context inside the
collectable object.  The "context" here is the value that gets passed
to the finalizer procedure (typically, this would be the foreign pointer
inside a finalizable struct object).

Then, add-to-gc-finalizer! can be used to add objects to the finalizer
and remove-from-gc-finalizer! can be used to remove objects from the
finalizer.

Separating the collectable object from the context prevents the
collected object from re-entering the live system through the
finalization procedure (it may set! some variable to it, for example
and that's not desirable).  Apparently, this makes ephemerons easier to
implement.
I found this through Taylor Campbell's comment on Andy Wingo's blog post:
https://wingolog.org/archives/2022/10/31/ephemerons-and-finalizers

Now I think this API is rather heavy-handed, but perhaps we can settle
on some sort of middle ground.  I also think that maybe separating the
finalization object from the "context" is too late for us, unless we
decide to rework the set-finalizer! API as well for CHICKEN 6.

Perhaps another API that exposes the queues directly might be better.
This way one could use one queue for different things, so that
finalization, signals and file descriptor readiness could happen on
the same queue, so that a thread could block on "any event".  A bit
like the waitForMultipleObjects API in Windows.

Something like:

(make-event-queue) => q

(enqueue-read q PORT)  ;; and a low-level (enqueue-read/fd q FD)?
(enqueue-write q PORT)
(enqueue-i/o q PORT)   ;; read or write
(enqueue-finalization! q OBJ)

(wait-for-event! q)  ;; blocks for any "enqueued" event
(peek-event q)       ;; returns #f if nothing ready

This makes it easy to pick off events from a queue, regardless of the
type and you'll get an object of the type.  I don't know if it can be
done easily, but it would allow GUI libraries to hook into this too
by having an event loop provide events into a queue that a Scheme
thread waits for.

Perhaps this was your goal all along for CHICKEN 6?  The scheduler would
need to be modified to make poll() put events on a queue instead of
having an i/o state per thread.

Anyway, sorry for the brain dump.

Cheers,
Peter

signature.asc
Description: PGP signature