Re: [PATCH] thread-safe handling of asynchronous events

[felix . winkelmann]

> On Wed, Jul 05, 2023 at 03:28:54PM +0200, felix.winkelmann@bevuta.com wrote:
> > The first patch provides the event-queue mechanism and cleans up the
> > scheduler a bit (hiding internal variables while also exposing ##sys#...
> > procedures to access them). This also defines hooks that a threading
> > API should override to allow suspension/resumption on events.
>
> I'm not super happy with exposing ##sys#fd-list and ##sys#timeout-list as
> a procedure under the exact same name that originally held a list.
> If there's any code that uses it, that would lead to strange errors.
> Might be better to either not expose it at all, or use a completely
> different name.

Yes, there is a potential for confusion. Different names might probably
be in order.

> > The second patch provides a new signal API: "make-signal-handler",
> > which creates a handler for one or more signals and "signal-ignore" and
> > "signal-default" (to ignore a signal or set the default disposition).
> > This is roughly modelled after the Racket API. The old "signal-handler"
> > and "set-signal-handler!" procedures have been deprecated.
> >
> > The third patch provides "make-finalizer", inspired by Chez' guardians,
> > but slightly different to allow blocking/non-blocking tests for
> > finalizations. The old API is still available, as often finalization
> > performs only very basic operations independent of the currently
> > executing context. Finalization procedures can be composed, as a
> > finalizer itself becomes subject to finalization once all associated
> > objects have been collected. Building guardians on top of this
> > should be straightforward, but the latter does not (to my knowledge)
> > allow blocking a thread until a finalizer triggers so I chose not
> > to implement this interface.
>
> This API looks nice, but as I understand it, make-finalizer requires
> all objects in each "pool" to be known when calling it.  AFAIK this
> means that when you are creating objects on-the-fly, you'd have to
> call make-finalizer N times, and when retrieving objects you'd have
> to poll N finalizer "pools", and therefore you can't block on all
> objects you'd want to collect.

That is indeed a shortcoming of the API. I must say that I'm not
too keen on the "guardian" approach, though, or your suggested hacky
workaround. There would be too many meanings attached to the
argument for the polling procedure. I also don't like the MIT
apporach, since our very basic "set-finalizer!" has worked fine so
far (with the execution context being the main problem).

> Separating the collectable object from the context prevents the
> collected object from re-entering the live system through the
> finalization procedure (it may set! some variable to it, for example
> and that's not desirable).  Apparently, this makes ephemerons easier to
> implement.
> I found this through Taylor Campbell's comment on Andy Wingo's blog post:
> https://wingolog.org/archives/2022/10/31/ephemerons-and-finalizers

I don't quite understand why many APIs are so afraid of retaining
the finalized object. The point is becoming aware of the object
being reclaimable. If it survives yet another GC cycle, so what?
Unless we are severely memory-constrained and must at all costs release
the object, I see no need in separating object and context for
finalization. Moreover, there may be cases when we may need full access
to the object to perform our finalization action.

> Now I think this API is rather heavy-handed, but perhaps we can settle
> on some sort of middle ground.  I also think that maybe separating the
> finalization object from the "context" is too late for us, unless we
> decide to rework the set-finalizer! API as well for CHICKEN 6.

Indeed. I wouldn't want to touch the underlying machinery. it works
pretty well, I think.

> Perhaps another API that exposes the queues directly might be better.
> This way one could use one queue for different things, so that
> finalization, signals and file descriptor readiness could happen on
> the same queue, so that a thread could block on "any event".  A bit
> like the waitForMultipleObjects API in Windows.
>
> Something like:
>
> (make-event-queue) => q
>
> (enqueue-read q PORT)  ;; and a low-level (enqueue-read/fd q FD)?
> (enqueue-write q PORT)
> (enqueue-i/o q PORT)   ;; read or write
> (enqueue-finalization! q OBJ)
>
> (wait-for-event! q)  ;; blocks for any "enqueued" event
> (peek-event q)       ;; returns #f if nothing ready
>
> This makes it easy to pick off events from a queue, regardless of the
> type and you'll get an object of the type.  I don't know if it can be
> done easily, but it would allow GUI libraries to hook into this too
> by having an event loop provide events into a queue that a Scheme
> thread waits for.

That sounds good, and I understand the motivation but I don't want to
touch the scheduler, to be honest. The UTF transition is already enough
work as it is. The original subject is merely a question of API choice,
let's not try to fix everything right now.

I will try to come up with something better, to allow adding objects
to an existing finalizer.


felix