[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] Add current-suspendable-io-status parameter

From: Mark H Weaver
Subject: Re: [PATCH] Add current-suspendable-io-status parameter
Date: Tue, 14 May 2019 16:22:09 -0400
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux)

Hi Nala,

Nala Ginrut <address@hidden> writes:

> On Tue, May 14, 2019 at 7:01 AM Mark H Weaver <address@hidden> wrote:
>> I guess what you want is the ability to see incremental reports on the
>> progress of your large I/O operations.  Is that right?  If we are going
>> to add an API for this, it needs to be reliable, and always give reports
>> in terms of the high-level requests that the user gave.
> Yes, that's exactly what I want. We need to get the progress of I/O
> operation when it's blocking
> so that we can compute a fair priority for the tasks.
>> My preferred approach would be something like this: we could add a
>> 'put-bytevector-some' I/O primitive which writes some bytes from a
>> bytevector, blocking only as needed to write at least one byte.  It
>> would return the number of bytes written.  This can be used to implement
>> an efficient variant of 'put-bytevector' that gives you access to the
>> real-time progress information.
> I'm not sure if put-bytevector-some does the work, I'll list my concerns:
> 1. All I/O will be managed by Guile when we enabled suspendable-port.
> That is to say, from the users side, users never know their I/O
> operations are blocking or not. It's transparent to users.
> Guile will guarantee the I/O operations to be finished by managing all
> the blocking I/O mechanisms.
> Users can only interact with the task with read or write waiter, which
> are registered by users themselves.
> In this scenario, users are out of control of I/O operations. And they
> have no way to get the progress of I/O, since there's
> no way to pass this status to the waiter function except for
> parameters in my patch.
> 2. suspendable-port module has already provided a bunch of overridden
> bytevector-* functions.
> However, they're hidden from users. I think it's good since the
> purpose of suspendable-port is to abstract all these details from
> users. Users only consider the read-waiter and write-waiter for scheduling.
> If we provide the low-level bytevector functions to users to let them
> do the non-blocking I/O by themselves, just like most C framework
> does. Then Guile suspendable-port will lose a critical feature,
> although users can still implement asynchronous non-blocking I/O by
> themselves with a non-managed approach. Say, do the I/O, check result
> by themselves, and do the scheduling.
> Personally, I'm fine with this way, since I'm familiar with both ways.
> But managed I/O of suspendable-port is a good selling point for many
> inexperienced server-side developers, they can use it in Scheme just
> like IOCP or AIO.
> Of course, I may misunderstand your mind.
> Could you elaborate more about your approach?

Here's what I had in mind.  I'm not suggesting that you use asynchronous
non-blocking I/O.  'put-bytevector-some' would be like 'put-bytevector'
except that it can write less than you asked it to.  Ideally it would
block only as needed to write at least one byte.  It would be analogous
to POSIX write(2).

Here's an untested draft implementation of 'put-bytevector*' which sets
the parameter 'current-io-status' to provide the information you wanted,
except that here the information will reliably refer to your own buffer.
The primitive it is built on 'put-bytevector-some' does not yet exist,
but we could add it.  Note that 'put-bytevector-some' would still block,
and it would be suspendable.

--8<---------------cut here---------------start------------->8---
(define* (put-bytevector* port bv
                          (start 0)
                          (count (bytevector-length bv)))
  (let loop ((start start) (count count))
    (unless (zero? count)
      (let ((written (parameterize ((current-io-status (cons start count)))
                       (put-bytevector-some port bv start count))))
        (loop (+ start written) (- count written))))))
--8<---------------cut here---------------end--------------->8---

We can already do something analogous on the reader side, using either
'get-bytevector-some' or the already-implemented-but-not-yet-pushed

What do you think?


reply via email to

[Prev in Thread] Current Thread [Next in Thread]