Further developments are now in, features are (guided somewhat by how the memory in linux/unix is managed and current guile strings)
1. chunked vectors e.g. we dived the vector in areas with different properties
2. optionally typed e.g. we consider a vector of u32 etc of data so there is a notion of object length and byte length
3. unaligned data transfers can be mitigated (controlled by a depth parameter) by considering a tree e.g. the chunk is a supervector itself with the same size as the old chunk
4. growing and shrinking can be effective
5. we have a pre side and a post side so queues and stacks are well modeled by this
6. fast reverse operation (useful for queues)
7. supervectors can be read only
8. ro super vectors allow cow (copy on write) sharing
9. types can grow e.g. we can start with a u8 general layout and have some chunks in u16 and some in u32 all tree like and controlled by an allowed depth
10. we can shar data that means that a shared copy can write to the original vector it copied from
11. zero chunks
12. all in scheme, but can take advantage of superfast c-code that mayby uses some advance processor op not possible in guile etc for hot paths, the advanantage
is that i) the chunks means better fibers integration with no long stalls because it repeats frequently to scheme and ii) the supervector library have an API
that makes the programming experience pleasant to implement the operations.
13. good default sizes of chunks to make sure that they are large enough to amortise the overhead of the chunks
14. fast append append!
15. pretty fast indexing, lookup of a position where the speed goes from basically a vector lookup to tree lookup controlled by the level parameter
16. auto aligns supervectors chunks to allow fast operations.
Next
a) testing testing testing and debugging
b) making a string implementation with guiles string interface that will guide some util functions for the general supervector.
c) try find a regexp library that can work with chunked vectors.
d) make the supervector work with ports so that we do not need to go via strings to create a large data structure and instead of the chunked one which is built
by smaller moderate sized pages
e) enable user defined data on chunks
f) some data transfers can be much faster in C when e.g. one copy from u8 to u64
g) allow supervectors with SCM objects as elements
h) allow chunked bitvectors and chunked bitvector operations
i) docs like readme
Project: