Rewrite of Guile's array system in Scheme

[lloda]

Hi,

This is to announce that I've been working on a rewrite of Guile's array 
system, entirely in Scheme.

        https://notabug.org/lloda/guile-newra
        https://github.com/lloda/guile-newra

Newra works on Guile 2.2, but it's quite a bit faster on Guile 2.9.

There are several reasons for this rewrite. A major one is that the current 
array system forces constant switches between C and Scheme and is opaque to the 
Scheme compiler. As the Scheme compiler improves this becomes more noticeable. 

The work is at a stage where all of the functionality of the old system is 
available in the new one, except for a few items that require patches to Guile 
itself (e.g. compiler support for literals, the C interface). These are listed 
at the end of the README. On the other hand the new system has a few extra 
whistles, such as basic rank extension / broadcasting and lazy index vectors.


This is still not a full featured array system, but it's a superset of what 
Guile offers now and hopefully it's a better base to build on.


There is a test suite and a benchmark. The documentation consists of the 
README, plus docstrings. The output of the benchmark on two different machines:

        debian i7-4930K - https://paste.debian.net/1094626/
        mac i5-8500B - https://paste.debian.net/1094627/

On Guile 2.9 some things are appreciably faster already, others are about the 
same. The only places where the C is still out of reach is where there is a 
fast path, such as array-copy! between #t types, which ends in a memcpy() in C.

--

The reason I announce this now is that I'm at a point where some design 
decisions are needed — the old API is inconvenient and we should have something 
that looks closer to they typical array API. I also could use some help with 
optimization. Of course a review or any other kind of feedback would be welcome 
as well.

To be clear, it is my intention, in due course, to propose Newra for inclusion 
in Guile.

Regards

        - Daniel