Re: [Nmh-workers] Slowdown with a Populated Folder.

[David Levine]

Ralph wrote:

> My rmmproc creates filenames like ,10139,1388316255.094920554
> so as more of those pile up, less entries fit in the 32KiB and
> more calls to it are needed and there's more entries starting
> with `,' to be skipped over.  Clearly, I need to switch to ','
> + base254-encoding.  ;-)

Or move the ',' files to a separate directory?  That would just
delay your problem, but maybe significantly?

> A couple of points come from it...  Why does mark(1) above have to read
> the folder when it's just doing a list of the .mh_sequences?

Well, it doesn't have to, but this call sequence is used in about
30 programs:

    call folder_read()
    call m_convert() to parse message ranges and sequences

including mark(1), which then goes on to add, delete, and/or list as
requested.

If we had to do it all over again, we'd likely come up with a
different folder data structure that better favors time over
space.  That would be an interesting project, but I'm not about
to take it on.

> Is it verifying the sequence's content against what still sits in
> the folder?

It is, given the sequence above.  If we expect that most
sequences have small numbers of messages, then there's
clearly a more efficient way.

> `mark -s foo -d all' gives an error if sequence foo
> doesn't exist, at least with this old version.

And it still does.  mark hasn't changed significantly since
1.3 (and beyond, I expect).

> Consequently, another mark is needed first, the one above, to
> detect if the sequence exists.  A more useful interface would
> cut the number of userspace commands that need to run, and
> thus the number of getdents.

Or is there a way to deal gracefully with the error if
sequece foo doesn't exist?

David