Re: wip-ports-refactor

[Andy Wingo]

On Wed 11 May 2016 16:23, address@hidden (Ludovic Courtès) writes:

> Andy Wingo <address@hidden> skribis:
>
>>                        | peek-char | read-char | peek-byte | read-byte
>>   ---------------------+-----------+-----------+-----------+----------
>>   2.0                  | 0.811s    | 0.711s    | 0.619s    | 0.623s
>>   master               | 0.410s    | 0.331s    | 0.428s    | 0.411s
>>   port-refactor C      | 0.333s    | 0.358s    | 0.265s    | 0.245s
>>   port-refactor Scheme | 1.041s    | 1.820s    | 0.682s    | 0.727s
>>
> My current inclination, based on this, would be to use the
> “port-refactor C” version for 2.2, and save the Scheme variant for 2.4
> maybe.
>
> This is obviously frustrating, but I think we cannot afford to make I/O
> slower than on 2.0, where it’s already too slow for some applications
> IMO.
>
> WDYT?

Humm, you might be right! I still have some questions though.

Before the questions, one point -- the "port refactor C" and "port
refactor Scheme" variants use the same underlying port structure.  The
Scheme variant just implements part of the port runtime in Scheme
instead of using the C runtime.  So, it would be possible to have the
Scheme version in a module in 2.2, if that were useful -- and I think
it's useful enough to enable green threads that suspend on I/O.

Regarding speed, you say 2.0 is too slow, but I could not reproduce that
in my initial benchmarks.  But, I can't be sure what you mean -- there
are the different axes as to whether you're processing input a byte at a
time, or in a block of bytes, whether you are decoding text or not, and
whether the port is buffered or not.  Do you recall any more details?
In my experience I never found 2.0 I/O to be too slow, but your mileage
evidently varies.

On all of these different axes, wip-port-refactor will be better because
it uniformly buffers the input through a standard buffer, which I/O
routines like get-bytevector can peek into directly.  This is true
whether the I/O routines are written in Scheme or in C.

The cases that I measure above are the worst cases for the difference
between C and Scheme, because they have all the overhead of a potential
slow-path that does a buffer fill and BOM handling and what-not but they
do very little I/O (just a byte or a char) and have no associated
workload (they do nothing with those bytes/chars).  Routines that read
multiple bytes at a time would not have as big a difference between
Scheme and C.

Furthermore with the uniform buffer, we can rewrite things like
read-line to peek directly into that buffer instead of doing a bunch of
read-char operations.  This would be big news for things like the web
server.  We could make this refactor either in Scheme or in C but I
suspect the performance would be similar, and Scheme is better than C
;-)

But, probably it is time to step back a bit now that the core changes to
the ports infrastructure have been made and seem to be performant
enough.  I will see if I can manage to get the extra internal helpers
that I had to expose shunted off into a separate module that's not
exposed to (guile-user), and then if that's all looking good I'll
update documentation and NEWS and see about a release.

What think ye?

Andy