[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Big XML files... (was Re: [Pan-users] Re: Better processing of very

From: Steven D'Aprano
Subject: Re: Big XML files... (was Re: [Pan-users] Re: Better processing of very large groups?)
Date: Sun, 5 Jul 2009 00:02:02 +1000
User-agent: KMail/1.9.9

On Sat, 4 Jul 2009 01:43:55 pm Ron Johnson wrote:
> On 2009-07-03 22:15, Steven D'Aprano wrote:
> > On Sat, 4 Jul 2009 12:56:36 pm Ron Johnson wrote:
> >> Also (and maybe because I'm a DBA), this problem just *screams*
> >> for SQLite and a database in the "First Normal Form".
> >
> > Please no. SQLite has problems with NFS. It's not so much that
> > SQLite won't work on NFS as that when (not if) something breaks,
> > it's difficult to unbreak it.
> Honestly, how many people who use Pan also mount $HOME over NFS?

Most people using a home directory mounted on a central server. That 
means (1) techies; (2) family and friends of techies; (3) school and 
college users; (4) enterprise users; etc.

Similar problems, for much the same reason, affect users who mount their 
home directory over Samba, and Mac users who use AFP. I don't know if 
it effects Windows users who mount their profile over native Windows 
SMB, but I wouldn't be surprised if it did.

> Obviously (as in your use-case below), some people do.  But they are
> mostly in "institutional" settings, where Pan probably wouldn't be
> approved anyway.

So what you're suggesting is that it's okay for Pan to rely on a piece 
of software with a known failure mode, because most users who will be 
at risk of that failure won't be allowed to use Pan anyway.

Apart from the ones who are allowed, of course.

> > My employer supplies Linux desktops to prisons. We have to support
> > a browser, and the system uses NFS. After an unexpected desktop
> > shutdown, to recover Firefox 2 you just need to remove a couple of
> > lock files. For Firefox 3, we needed to open the database
> > (read-only) with the sqlite3 command-line tool, dump the data into
> > a new sqlite database, quit, replace the "locked" file with the new
> > "unlocked" file.
> That's poor transaction handling on the part of Mozilla, unless
> SQLite is *sooooo* broken that it corrupts it's data file even when
> no transactions are active.

As far as I know, the data isn't corrupted -- it's just locked, and no 
other process can unlock it. If the original process is gone, Firefox 3 
doesn't work right because it can't access your profile.

Here's a related bug:

It contains links to many similar and related bugs. With so many bugs 
being related, and marked as duplicate, it's hard to tell exactly what 
the status of the issue really is. The above is marked as fixed, but it 
seems to refer to a specific issue related to sqlite locking and not 
necessarily the one I'm referring to.

Here's another sqlite-related bug:

Looking at the trouble the Mozilla developers have had with unforeseen 
(because they didn't read the FAQs?) side-effects of using SQLite, for 
the minimal benefit they've had, my opinion for what it's worth is that 
they would have been better off going for a less heavy-weight solution 
to the problem of "My bookmarks are corrupt!!!". I mean, honestly, ACID 
compliance for your browser history??? What the hell are they thinking? 
When the user asks Firefox to *explicitly* save a file, it's at the 
mercy of the file system whether or not it gets written to disk, but 
your history and other privacy-compromising footprints get treated as 
critical data. Crazy.

Steven D'Aprano

reply via email to

[Prev in Thread] Current Thread [Next in Thread]