Re: [Pan-users] Re: Can creating task list be optimized?

[Darren Albers]

Duncan wrote:
> I just saw how smoothly it was going with the big-post multi-part group,
> and having just bought 6 months worth of newshosting, when I entered a
> thousands of small-posts single-part group, I saw that money I just spent
> on it going up in smoke, or more precisely, in bottlenecked CPU cycles. 
> That's /extremely/ frustrating, and it's absolutely true, klibido was
> managing full speed in a similar group (tho with less posts as I had only
> Cox at the time) and taking less than half a CPU to do it, so pan
> bottlenecking on one CPU and as a result doing barely dialup some of the
> time... very very ungood!
I think Charles may have fixed this as well, from the bug report:
--------------------------------------------------------------------------------------

The patch is too big to attach here, but for 0.110:

* inserting 25k articles into the queue now takes three seconds,
 compiled without optimizations, on my four year old AMD budget box.

* with 25k articles in the queue and 12 connections (3 servers x 4 per server)
 grinding away, Pan's averaging 15-30% CPU on the same budget box.

* Tweaked away some small amount of memory overhead in Task object
 by converting a lot of std::set to Loki::AssocMaps.  (i.e., for
 unchanging containers that   need to be bsearched frequently,
 use a sorted array instead of a btree
 to avoid node/pointer/fragmentation overhead).

Thanks for reporting this, Samuli.  This is exactly the kind of
performance that Pan needs to be able to brag about to show
why the rewrite is better than 0.14.x. ;)
--------------------------------------------------------------