Re: [Pan-users] Policy discussion: GNKSA

[Rhialto]

On Mon 04 Jul 2011 at 20:03:54 +0000, Duncan wrote:
> Another way around it that would help for some, would be to move the 
> decoding/saving to separate threads, so the download threads could 
> immediately go back to downloading.  However, results there would vary, 
> depending on local computer resources (single-core and speed vs. dual vs. 
> quad vs. 6 vs. 8/12/whatever, and local storage speed, high-speed RAID-0 
...

Maybe this is the place to add an observation that I could (should) have
made before. Nobody else mentioned it so I may be the only one who sees
it. It probably depends in some way on the implementation of threads on
the system where Pan is running, and I don't use Linux but NetBSD, which
is probably different from most of you.

What I see sometimes is that the task queue seems to get "stuck" at the
decoding stage. That is, some file is downloaded fine, and ready to be
decoded, but it isn't. It just stays in that state. In the mean time,
the next file is getting downloaded (usually). This way, the downloading
can get ahead of the decoding by many many files. Even by more of the
cache size. (Although I get the impression that files aren't thrown out
of the cache if they are queued for decoding... but I never checked the
code to make sure). At some point the decoding somehow decides to start,
and then its frenzy of disk activity may interfere with the frenzy of
the incoming/downloading data.

If this happens, especially with the last files in the task list, this
will definitely increase the wall-clock time.

I've been hoping that this bug would disappear "by itself". I have seen
it less lately, but I don't know why. (It'll probably take revenge by
popping up right away on my next download...)

-Olaf.
-- 
___ Olaf 'Rhialto' Seibert  -- There's no point being grown-up if you 
\X/ rhialto/at/xs4all.nl    -- can't be childish sometimes. -The 4th Doctor