Re: New egg: CHICKEN Transducers

[Jeremy Steward]

On 1/5/23 04:22, Chris Brannon wrote:
> Jeremy Steward <jeremy@thatgeoguy.ca> writes:
>
> > And I've written a short blog post outlining some of my frustrations
> > that led me to writing this egg:
> >
> > <https://www.thatgeoguy.ca/blog/2023/01/04/reflections-on-transducers/>
>
> That was a fantastic post.  I'm still digesting it.  How about SRFI 42?
> Here's your example from the transducer benchmark, written with SRFI 42:
>
> (import srfi-42)
> (time (fold-ec 0 (:range x 100000000) (if (odd? x)) (* x 3) +))

Thank you as well for the kind words. I am glad you enjoyed reading it.

You are not the first to bring up SRFI-42! There was someone on Mastodon 
earlier who made a similar point. I don't think it's a bad library, but 
it seemed to have a handful of things going against it:

1. Locked in stone: same argument for SRFI-171 really, although SRFI-42 
is a lot older and more battle-tested.

2. Macro-based: This makes it pretty difficult to extend, especially if 
you don't have a clear "standard" or "winning" data structure that the 
community can rely on. Not that it's impossible, but I'd have an easier 
time as a user writing a custom fold procedure (and maybe a collect?) 
and working with transducers than extending eager comprehension macros 
to support new types.

A corollary to that: transducers are a bit more general because they 
separate the collection from the primary form (transduce, or fold-ec in 
your example). This isn't really a strong argument to dismiss it though.

3. Aesthetics: related to syntax / macro based solutions, I have 
strongly religious (read that as: entirely subjective) reasons to prefer 
to keep my Scheme code mostly syntax free. Transducers are a bit more 
"functional programming" in some ways (we still use set! though?), which 
is acceptable to me.

To put it in perspective: if the SRFI community said "we're not writing 
new fold operations, no more (f k x) nonsense, and everyone has to learn 
SRFI-42 or write their procedures in their own Scheme modules" I'd 
probably still find that acceptable. The thing is that eager 
comprehensions aren't really pointed out or taught or talked about 
really all that often, especially with regards to newbies learning the 
language, or to try and guide "best practices" in Scheme. Of course, we 
don't want to tell people too much how to program, but I think 
encouraging code that is more performant or easier to "fix" (or debug) 
when things go wrong is probably better than not.

So if I had to decide between transducers and SRFI-42, I'd be sad 
because both work well. But I'm happy that we can choose either. I just 
have a stronger preference for the transducers approach (and 
|collect-vector| variants are extremely underrated relative to how 
useful I believe they are).

> And here is a comparison of performance characteristics:
>
> Script started on 2023-01-05 03:14:57-08:00 [TERM="screen.linux" TTY="/dev/pts/4" 
> COLUMNS="80" LINES="30"]
> $ ./transducer-bench
> 12.278s CPU time, 0.002s GC time (major), 1/108400 GCs (major/minor), maximum 
> live heap: 879.61 KiB
> $ ./srfi-42-bench
> 3.5s CPU time, 0/70231 GCs (major/minor), maximum live heap: 478.02 KiB
> $ exit

Oof, I'm surprised to see such a differential. On my machine it's still 
faster, but it's:

transducers: 3.540s
srfi-42:     1.623s

Still though, looks like I have my work cut out for me.
--
Jeremy Steward