[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Pan-devel] Replace custom data structures with standard ones?

From: Rui Maciel
Subject: Re: [Pan-devel] Replace custom data structures with standard ones?
Date: Mon, 11 Feb 2013 13:51:46 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130106 Thunderbird/17.0.2

On 02/11/2013 12:51 PM, Duncan wrote:

Really, the sticking point is that pan constructs its entire tree in-
memory.  There has long been talk of doing a database backend using sqlite
or mysql or the like, such that pan only works with a few pages worth of
data in memory at once, the rest is in the db, but I think Charles wasn't
a db expert and hesitated to go there.  Now of course Charles has moved
on and it's Heinrich that has been doing most of the new features and
heavy coding of late.  He too has mentioned that he plans a db backend at
some point, but I've no clue whether he has a branch he's hacking on to
that end yet or not...

What I'm saying is... switching to standard data structures and etc may
have value, particularly if it's even better memory scaling than current,
but be absolutely sure you keep the scaling in mind, anticipating and
testing new code against sufficiently busy groups on servers with enough
retention to give you at least 200 million header groups to test against.

But any real serious data structure rewrite work should almost certainly
target a database backend solution of some sort as in reality, that's
about the only way to address the whole scalability thing once and for
all.  While I have no idea where Heinrich might be on the database backend
work, I'd guess he at/least/  has some rough ideas about implementation,
so it'd probably be a good idea to discuss that with him (and there's a
couple others who might be interested) before diving in head first.

I didn't had in mind such a fundamental redesign. Instead, it would only be about replacing a set of custom data structures with their C++-standard counterparts, using them as some sort of drop-in replacement. The main advantage would be that such a replacement would lower Pan's learning curve for those interested in contributing to it and doing some code maintenance, as no one would be required to parse the largely undocumented source code of these components to get a hold on how to do things Pan's way. Another advantage would be that it would be possible to trim down the source code tree by removing code which is almost pushing a decade by now, and apparently hasn't been maintained in the past half dozen years or so. And there's also the janitorial aspect of it: it's always nice to clean up the code and remove the cruft. In short, it would only be about replacing custom components with standard ones, and cleaning up things in the process.

Rui Maciel

reply via email to

[Prev in Thread] Current Thread [Next in Thread]