[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: New package: resist!
From: |
Alexandre Garreau |
Subject: |
Re: New package: resist! |
Date: |
Sun, 12 Dec 2021 02:59:38 +0100 |
Le dimanĉo, 12-a de decembro 2021, 2-a horo kaj 35:48 CET Michael
Heerdegen a écrit :
> Alexandre Garreau <galex-713@galex-713.eu> writes:
>
>
>
>
> > > How does your package handle values whose print syntax can't just be
> > > `read' to recreate the value?
>
>
>
> > that’s an issue for pretty much any persistence package, and it would
> > just as well be while using sqlite, persist, eieio-persistent or
> > anything else.
>
> Yes, but it's the hard part. It is what makes saving the Gnus registry
> slow, and it is what makes eieio-persistent look "like an object
> oriented prin1".
No, as said before you’re conflating two separated issues: variable
persistence, and element-by-element retrieving (from a kv store). We use
a kv-store for variable persistence, but that feature is different from the
later one because you save each value once in its entirety, and then fetch
it back once in its entirety. So reading or printing it is actually
necessary, you cannot make it faster, and it’s not a bottleneck (I/O is
the bottleneck then), you have to serialize anyway. When you want to
fetch individual elements without restoring the whole value in its
entirety (or perhaps doing just as if it was, in a lazy way, but that’s
yet again a new feature which has not been discussed yet, and which enters
in competition with the kernel’s paging/swaping features), you need to
destructure it manually, you don’t do that automatically, and you want its
restoration to be coordinated in some determined way… that’s not variable
persistence. So the problem you’re talking about, where “read” and
“write” are slow, is not variable persistence.
I feel like the features we came to discuss are so abstract people keep
confusing and conflating them together. They all in some way slightly
overtake one onto the other, so it makes it understanding overall even
more difficult…
> > Now we could try to see what would happen concretely: what kind of
> > values are you thinking to?
>
> The Gnus registry is one example. Apart from that: most values,
> actually, apart from trivial ones: lists, hash-tables, structs. Most
> applications will not just need to save integers and strings.
That’s a separate problem, again. These value are perfectly printable.
You just claim we can store them so that they’re more efficient to
(partially) retrieve. And that’s not a problem solveable by variable
persistence (since the concept of variable persistence is you restore back
the entire variable into memory), so it’s not even related to the current
feature being discussed.
If you want to solve that, you have to use the lower level feature we
discussed before, a kv storage mechanism, to store your hashtable or
alist. A struct is by definition limited so there’s little interest in
storing it strangely (but you store an array of them you may want a
relational database, indeed… except until now we only discussed about
needs of a simple kv storage, which is lower level than a relational
database). A *general* list, or an array, may be amenable to a hashtable/
alist with succesive increments as keys, but if you have any more adapted
idea I’d like to hear it (because otherwise it looks a little overkill).
But you then have to *manually* prepare a store for that, and manually
split up your sequence into that store.
We cannot apply that to the issue discussed now, variable persistence,
hence any variable, because that would mean simply recursively make all
lisp objects which are sequences stored into kv stores (because, as you
know, a hashtable can contain another hashtable, and two sequences can
share elements), that’s just overkill, it consists in changing the entire
elisp runtime, and in the end you just replace every variable with a query
to a relational database… which is interesting, but has not been discussed
until now. You would make one table for each type, and pointers would be
replaced by foreign keys. Is that what you want? is that what you’re
talking about? That would be the “most efficient way” if we wanted to store
all emacs ram to disk, and yet be able to retrieve it while being sure to
do it most efficiently wrt what a relational database can do, with the least
reading from disk and the least stuff kept into memory.
The ideas in this discussion start getting very weird… now I’m wondering
if it would be possible, in a perfect world, to at the same time describe
perfectly the native way of storing variables/(PI/repositionable)pointers/
number in memory so it can directly be dumped to disk and portably be
decoded at will, and design a relational database format so that it can be
mmaped to memory and be used to represent variables and complex objects
directly (or the otherway around: encode, and defragment/fragment/group
(through action of a special garbage collector) objects into memory so
that if it’s dumped to disk it gives a most efficient relational database
data, and yet be effecient to read (like data often used together would be
near each other)), and use that to represent all data on a computer, for
every programming language/runtime/software…
- Re: New package: resist!, (continued)
- Re: New package: resist!, Michael Heerdegen, 2021/12/11
- Re: New package: resist!, Alexandre Garreau, 2021/12/11
- Re: New package: resist!, Michael Heerdegen, 2021/12/11
- Re: New package: resist!, Qiantan Hong, 2021/12/11
- Re: New package: resist!, Michael Heerdegen, 2021/12/11
- Re: New package: resist!, Alexandre Garreau, 2021/12/11
- Re: New package: resist!, Michael Heerdegen, 2021/12/11
- Re: New package: resist!, Alexandre Garreau, 2021/12/11
- Re: New package: resist!,
Alexandre Garreau <=
- Re: New package: resist!, Qiantan Hong, 2021/12/11
- Re: New package: resist!, Michael Heerdegen, 2021/12/12
- Re: New package: resist!, Qiantan Hong, 2021/12/11
- Re: New package: resist!, Qiantan Hong, 2021/12/11
- Re: New package: resist!, Dmitry Gutov, 2021/12/10
- Re: New package: resist!, Stefan Monnier, 2021/12/10
- RE: [External] : Re: New package: resist!, Drew Adams, 2021/12/10
Re: New package: resist!, Stefan Kangas, 2021/12/09