[Pan-users] Re: Question around multi-server and connection errors

[Duncan]

"Brian King" <address@hidden>
posted address@hidden,
excerpted below, on  Sun, 29 Oct 2006 02:20:56 +0200:

> I'm seeing something this behaviour, too.  It seems to happen for (from my
> point of view) random binary articles.  The ones that are broken are
> really broken - watching it in the task manager, it says that it's 100%
> complete, but it's shows a time left and kB/s.  When the time gets down to
> a certain point the task gets requeued and the time left increases to
> where it was before.  This repeats indefinitely.

I've seen a number of folks using other news clients complaining about
this.  Apparently, someone's deliberately poisoning the well, so to speak,
replacing good posts with corrupted ones.  Since the corruptions don't
match the yenc crc-32s or sizes, some clients, including pan it would
seem, get mixed up.

There is or was a similar problem due to a bad transit server corrupting
certain posts.  Apparently, giganews was one of the few servers to have
the uncorrupted versions of most of these.  I haven't tracked it myself,
but this is what a couple guys on my ISP's private newsgroups have been
saying, so I'm forwarding it here as it seems appropriate.

I'm not sure if what I've seen has been one of those problems or something
not quite as bad, but using newshosting, enough has been coming thru to
handle it fine in most cases, and where it doesn't, if I manually save the
/text/ (NOT the attachment) of the affected posts, then decode with
uudeview (which handles yenc also), it recovers all but a few par2 blocks
worth of the files, enough to easily par2-repair the rest, generally with
less than half of the provided par2 repair blocks, often with only one or
two needed.

Do note that I routinely download to cache (which is set accordingly high,
I use a 12 gig dedicated partition, edit preferences.xml directly to set
cache size) first, then process the files, sorting and saving or deleting,
from there.  Doing it that way, it downloads what's there to download,
without trying to decode or save the attachment, saving the decode for my
sorting/saving step. Thus, it completes the download and moves on, not
knowing or caring about the bad decode or yenc crc-32 value until later. 
Since the stuff's all in cache when I go back to it, if it won't save the
attachment, I can simply save the posts as text to a scratch location and
decode them manually.  As long as the files were par2-ed so I can repair
the few missing blocks, I haven't had that technique fail me yet.  Often
a few of the parts will be damaged, but I'll only be missing a couple
blocks when I check it, so it's obviously recovering most of the bad file,
even where pan wouldn't decode and save it on its own.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman