Re: [Pan-users] Re: Big XML files... (was Re: Re: Better processing of very large groups?)

[Steven D'Aprano]

On Sun, 5 Jul 2009 12:43:18 pm Ron Johnson wrote:
> On 2009-07-04 21:23, Steven D'Aprano wrote:
> > On Sun, 5 Jul 2009 05:23:20 am Ron Johnson wrote:
> >> On 2009-07-04 13:57, Matej Cepl wrote:
> >>> Ron Johnson, Fri, 03 Jul 2009 21:56:36 -0500:
> >>>> Also (and maybe because I'm a DBA), this problem just *screams*
> >>>> for SQLite and a database in the "First Normal Form".
> >>>
> >>> After reading http://www.jwz.org/doc/mailsum.html and having
> >>> still alive experience with Evolution,
> >>
> >> Corrupt that mbox file and *poof*, there goes years of email.  I
> >> stopped using it years ago as anything but a bzipped archive
> >> format.
> >
> > Yes and no ... you've still got the emails, in text format, so I
> > suppose
>
> Not if the write fails in mid-stream.  

Well I suppose that's one failure mode which could lose the entire 
mailbox.

The only failure I've experienced with mbox was one where either the 
first two or three, (or possibly the last, it's been a while and I 
forget) emails in the mailbox got corrupted. I could still see their 
content when I opened the file in a text editor, but in the mail 
clients they showed up with blank headers and no body.


> Remember the recent kerfuffle 
> regarding KDE assuming that the way ext3 works is hows every file
> system works, and thus losing config files on ext4 partitions?

No. Got a link?



> > you could write a recovery utility, if one doesn't already exist.
> > But yes, I agree, maildir is better than mbox because you're likely
> > to lose no more than one message in the event of corruption. But
> > keep in mind that when Netscape 2 came out, mbox really was the
> > standard -- these days I'd say only old dinosaurs use mbox.
>
> 90+% of the people using Tbird still use mbox...

Like I said, old dinosaurs.

A few months back, I broke my Kmail config, and decided I'd check out 
Thunderbird (I haven't used it since it was part of Netscape 3). I gave 
it a good go, I really did, but it felt like I was being asked to do 
carpentry with my hands cuffed together and a small monkey riding on my 
back hitting me with over-ripe bananas.

I wouldn't say it is unusable, but I think it's aimed at users whose 
expectations are lower than mine.

That's not to say that Kmail doesn't have its problems too... all 
software sucks, it's just that some sucks less than others.


> > And at least mbox is a text format, and you have one file per mail
> > box, and not one giant undocumented binary file for all mail boxes
> > like Exchange uses.
> >
> > *shudders*
>
> It's the undocumented part that disturbs me.

If you're worried about mbox corruption losing an entire mail box, you 
should be worried about Exchange corruption losing every mail box for 
every user.


> I'm also sure that "they" really screwed the pooch when designing
> the PST file format.  The Outlook XP format, though, does seem to
> perform better than the older version.

Yeah, but that's not saying much.



-- 
Steven D'Aprano