Re: [Pan-users] Big XML files... (was Re: Re: Better processing of very large groups?)

[Duncan]

Steven D'Aprano posted on Sun, 05 Jul 2009 12:56:01 +1000 as excerpted:

>> 90+% of the people using Tbird still use mbox...
> 
> Like I said, old dinosaurs.
> 
> A few months back, I broke my Kmail config, and decided I'd check out
> Thunderbird (I haven't used it since it was part of Netscape 3). I gave
> it a good go, I really did, but it felt like I was being asked to do
> carpentry with my hands cuffed together and a small monkey riding on my
> back hitting me with over-ripe bananas.
> 
> I wouldn't say it is unusable, but I think it's aimed at users whose
> expectations are lower than mine.
> 
> That's not to say that Kmail doesn't have its problems too... all
> software sucks, it's just that some sucks less than others.

OK, I'm rereading this thread based on the current sqlite solution 
discussion, and...

What do you think of the new, akonadified (and thus sqlite, mysql, or the 
experimental postgresql backends) kmail?

FWIW, I had maintained a wait-and-see attitude thru 4.4 and 4.5 with the 
kaddressbook switch to akonadi (for 4.4), waiting to see what kmail2 
would be like with its switch.

When kdepim 4.6.0 and 4.6.1 came out, with the new akonadified kmail2, I 
tried them/it.  Suffice it to say that after 9 years on kmail (since kde2 
era), with MSOE archives imported into kmail when I switched that go back 
another five or so...

I switched to claws-mail and couldn't be happier.  After setting up a 
second claws-mail instance with its feed-reader plugin (wrapper script to 
set a couple variables to keep them separate, I don't really want my mail 
and feeds in the same app-instance, but when the app works as well as it 
does, different instances of the same app, same general UI and hotkey 
setup, etc, are nice) to replace akregator as well, I was able to totally 
kill kdepim and eliminate the scourge from my system!

The data conversion from kmail wasn't easy, both the addressbook and 
message import scripts were a bit stale and needed a bit of tweaking to 
work, but they did, and I had to convert my 50-ish mail-filters by hand 
as I couldn't find a script for that, but the experience wasn't really 
worse than the kmail upgrade conversion experience, which was buggy too 
even tho they'd essentially frozen kdepim for an extra year to work on 
it.  And unlike the kmail upgrade, the claws-mail conversion was actually 
worth it, and *THEN* some! =:^)

I guess I can count myself lucky that kmail proved a useful solution for 
so long, but eventually it had to end.  Hopefully claws remains equally 
useful (actually, it's already more so) over an even longer period.


What I'd like to know tho and what's of interest for this list, is how 
claws-mail, with its mh-format (precursor to maildir, single plain-text-
message per file) mail storage, maintains the speed and slim memory 
footprint it does.  Pan and claws-mail actually have similar data store 
sizes here, but there's no comparison in especially cold-system-cache 
startup speed.

What sort of indexing, trees, and message-thread-processing does claws-
mail use to be so fast and memory-slim, and what can pan learn from it 
for its own use?

One thing claws-mail seems to do is single-thread its mail downloads, 
serially checking each configured account in turn.  Obviously, that's not 
going to work for pan, but pan could sure learn from however claws-mail 
handles its startup, since claws-mail starts in seemingly no-time, 
certainly compared to pan with an 800 MB cache.

And claws-mail isn't using sqlite or similar database to manage things, 
either.  In fact, that's one of the reasons I dumped kmail once it 
akonadified.  I didn't need a singing, dancing, combined database kontact 
sub-app like it seems everything kdepim related is becoming; I just 
wanted an email app that handled mail, and did it well, without all the 
unnecessary bells and whistles and singing and dancing, as that only got 
in the way (and made the app *WAY* more complex and unreliable) for what 
I needed it to do.

And claws-mail does what I want and more, plain-text, highly customizable 
(its hotkey dump is about twice the size of pan's accels.txt, for 
example, and its external command invocation allows scripting actions, 
filters, etc, if it doesn't have a builtin solution for what you want), 
and it's **FAST**!!

I've actually setup a third claws-mail instance, for news, too.  But 
unfortunately, unlike claws' mail functionality, the news functionality 
is barely documented, and from my limited experiments so far, it's way 
more limited than pan, tho I still might eventually end up switching my 
text groups over, basically gmane, which is all I tend to use pan for 
these days anyway as I don't have a paid news-provider account ATM.  I'd 
still keep pan around at least initially, in case I ever got back into 
binaries, but given claws-mail's speed on mail at least and memory 
footprint, if it's even close to that for text groups, it could well be 
worth the switch for them.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman