Re: New egg: CHICKEN Transducers

[Jeremy Steward]

On 1/5/23 05:49, siiky wrote:
> Awesomesauce, thanks for the egg and the post! This is a smnall-ish pain
> point of mine too. I'm still processing the post as well, but I can tell
> I'll start using this frequently!

That's great to hear. I appreciate that you enjoyed the post and will 
start using transducers.

> First a couple of typos: In footnote 4 it should be `call/cc` and not
> `call-cc`? And "its not the Scheme I reach for" :p

Oof. Thanks for doing the editing work for me. It should be updated now.

> About question (1) of "Some open problems ...": IMO (transduce type-fold
> ...) is a good compromise, and it seems better to me than
> (type-transduce ...) though it means a tiny bit more typing. My
> reasoning is that (intuitively, without having looked at an impl of
> either) type-fold is easier to implement than type-transduce, and thus
> it should also be easier for someone to extend the transducers egg with
> their own fold+collect rather than transduce+collect.  An usecase of
> extending the transducers egg is e.g. if I have an egg where I define a
> data structure foo and would like users of my egg to use transducers on
> (with?) foo, but without having to change the transducers egg itself.

Indeed, and this has been the reasoning in why I have left it for the 
most part. An egg like this is nothing if it cannot be extended easily. 
Noticing the distinction between implementing fold, collect, or some 
entire abstraction that tries to do both was something I realized about 
halfway through writing it (specifically, when I started adding fold / 
collect procedures outside of the list variants).

There is always the possibility to use the wrong fold with the wrong 
datatype or some-such, but there's really no way for me to get around 
that without a full object system and types, so ¯\_(ツ)_/¯

> Regarding (4) and strings: if we survived without transducers until now,
> I think can survive until transducers for strings until C6 too and use
> string->list+list-fold :) (I'm assuming UTF-8 will only become core in
> C6, maybe I'm wrong)

If there were enough of a strong signal to just go UTF-8 only I'd 
probably just do that. The problem is that transduction happens on an 
item-by-item basis and I'm not sure that any string-fold will work the 
way you expect it to. In my experience you don't want individual unicode 
code-points, you want the full character.

I guess if I had a special transducer (similar to chunks or 
chunks-exact) and a collector to take individual code-points and 
generate a full grapheme, then I could get something that approximates 
useful unicode transducers.

But as it stands, I think anything regarding strings is going to be 
hairy and icky, because locales suck and internationalization means that 
any string logic will always be (somewhat) fragile.

My kingdom for a day in which users can use strings in any locale as 
easily as some of us abuse ASCII.

> Small nitpick comment: I think it makes more sense to have either
> type-fold+type-collect or fold-type+collect-type. It's more common in
> Scheme to name type-operation so I'd give preference to that, but the
> other is more natural and equally good IMO.

I figured this criticism would come up. My thinking was that keeping 
collectors prefixed with `collect-` helps autocomplete and gives you an 
interesting symmetry:

    (transduce list-fold (compose ...) collect-vector (list ...))

Notice how you start with list- on the left and -vector on the right? 
The transducer is in the middle and so you have list-transducer-vector 
if you squint at it hard enough.

But that said: I'm completely willing to change that if I get enough 
people telling me they get it backwards and it's incoherent. Ping me so 
I can keep a count of how many times / how many individuals this trips up.

Regards, and thanks again for the comments, I really appreciate the 
feedback.
--
Jeremy Steward