Re: persistent data feature

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> There will, of course, be large amounts of data, and that will be slow.
>> (People already do this with a number things, and the results are
>> predictably sluggish.)
>
> To help me understand this discussion, I think it would help me to have
> examples of such large databases currently implemented as text files,
> along with an idea of what "large" means in this context (how many MBs)
> and where the "slow"ness manifests itself.

I can think of two examples of large text databases: elfeed and recent
org-mode's caching (org-persist).

The slowness manifests itself on reading/printing(saving) the database
and from the requirement to load the whole database into memory for
queries.

We are talking about 300Mb for elfeed and up to 20-30Mb per object
(usually below 1Mb though) for org-persist with 100-200Mb total object
size.

Best,
Ihor