Re: [Pan-users] Re: Big XML files... (was Re: Re: Better processing of very large groups?)

[Steven D'Aprano]

On Sun, 5 Jul 2009 05:23:20 am Ron Johnson wrote:
> On 2009-07-04 13:57, Matej Cepl wrote:
> > Ron Johnson, Fri, 03 Jul 2009 21:56:36 -0500:
> >> Also (and maybe because I'm a DBA), this problem just *screams*
> >> for SQLite and a database in the "First Normal Form".
> >
> > After reading http://www.jwz.org/doc/mailsum.html and having still
> > alive experience with Evolution,
>
> Corrupt that mbox file and *poof*, there goes years of email.  I
> stopped using it years ago as anything but a bzipped archive format.

Yes and no ... you've still got the emails, in text format, so I suppose 
you could write a recovery utility, if one doesn't already exist.

But yes, I agree, maildir is better than mbox because you're likely to 
lose no more than one message in the event of corruption. But keep in 
mind that when Netscape 2 came out, mbox really was the standard -- 
these days I'd say only old dinosaurs use mbox.

And at least mbox is a text format, and you have one file per mail box, 
and not one giant undocumented binary file for all mail boxes like 
Exchange uses.

*shudders*


> SQLite is "just" the obvious choice.  What happened to c-trieve, or
> any of the other b+tree libraries?

I think the point is that *any* database is (1) overkill for the 
requirements and (2) likely to lead to performance and corruption 
problems.




-- 
Steven D'Aprano