|
| From: | Glen Walpert |
| Subject: | Re: [Pan-users] Re: One group suddenly ignored |
| Date: | Sat, 09 Apr 2011 09:55:24 -0400 |
| User-agent: | Thunderbird 2.0.0.24 (X11/20101027) |
Thanks for the detailed suggestions, comments below. Duncan wrote:
OK, I will compile and install 0.134 as soon as I can finish some other work, and let you know the results.Glen Walpert posted on Fri, 08 Apr 2011 10:33:33 -0400 as excerpted:I have been using Pan 0.132 on Ubuntu 8.04 for a year with no problems until a few days ago when all posts to sci.electronics.design started being ignored ("get headers in subscribed groups" lists over 100 messages but they all disappear when the group is selected, no headers for the last 3 days even when no filters are selected so they are not just marked as read.) All of my other subscribed groups have no problems. If it is an issue with my news server (news.astraweb.com) it does not affect an old copy of Agent on XP, which still reads new messages on sci.electronics.design. I tried temporarily removing .pan2/Score with no effect (not a bad scoring rule) and reproduced the problem on a clean install of Pan 0.133 under Ubuntu 9.04 with no scoring rules. What else should I look for?First, note that pan 0.132 and previous had a security vulnerability that was fixed in 0.133. pan is obscure enough it's unlikely to be a problem unless someone's specifically targeting you, but you really want to upgrade that 0.132 instance so you're no longer exposed.It's also worth noting that 0.134 is out now, with various mostly bugfix patches that have been floating around for awhile. You may wish to consider upgrading to it, as it's possible the issue you are seeing is one of those bugs (see below for details). But for sure you want at least 0.133, due to that security issue fix!
As to your problem, there are three possibilities here. Well, four actually. I added this top (0) one later, on top, since it's easiest to deal with and get out of the way.Zeroeth, easy to test but too vaguely defined to know a fix immediately, report the results for further investigation. What happens if you use the get headers... dialog, and select get ALL headers? It's usual that pan might fill in the occasional missed post here or there, and they may or may not appear read, but does get all headers get the new ones that get headers in subscribed groups missed, or not? If it does, report that behavior and perhaps we can figure out what's happening. If not, well, we know it doesn't help, now. The reasoning here is that the get all headers function uses somewhat different methods to request them than the other options do, and it can at times get stuff that isn't picked up using the normal fetch new headers functionality. The problem in that case may well be a peculiarity of the server behavior, or a pan bug, or... but knowing one way or the other is more data on it than we have now.
Testing with 0.133 now:Get all headers retrieves a lot of headers, but does not get all the missing headers, and they are all marked as read! Even stranger, if I wait a while and get new headers (without deselecting sci.electronics.design), some new headers are retrieved, but they only show up in the headers pane when "show only unread messages" is selected!
First, easiest to fix, but probably not /your/ problem (unless you copies over the pan data sans scorefile from the old install to the clean install), and simply listed here for completeness, pan's read-message tracking depends on the server keeping its per-group message sequence numbers in order. If the server starts its numbering over, as it might do if it had a storage problem for the group and restored from a backup, or if you switch servers by just giving pan a new server address for an existing server config, instead of creating a new server config for the new server, the new message sequence numbers may appear to pan to be for old messages it long since expired, instead of the new ones they actually are.The fresh server config (I even uninstalled pan, deleted the .pan2 directory, and reinstalled with a new manual config) does not affect the problem.The fix for that is to either create a fresh server config so pan starts with a fresh idea of what the server message sequence numbers should be, OR, with pan not running naturally, to manually text-edit the appropriate newsrc file for that server (for multi-server people, the servers.xml file tracks which server uses which newsrc file), deleting pan's tracking of read-messages for that group, or simply delete that newsrc file entirely, if losing that info for all groups for that server isn't a problem.
Second, the problem alluded to above, fixed by an upgrade to 0.134. The problem here is that in huge groups, typically binary groups with tens of thousands of individual daily articles (a full ISO binary post may be split over thousands of individual articles for just that one post!), the message sequence numbers exceed the maximum number (a bit under 4.3 billion) that can be recorded with a 32-bit number! 32-bit pan especially (I'm not sure about 64-bit pan, BTW not /just/ pan, it was a problem for a number of news clients) used to use a 32-bit int for tracking message sequence numbers. The patch alluded to above switched pan to using a 64-bit number for the same thing. That's not going to run out in the foreseeable future! But that patch didn't make it into 0.133, only 0.134. Thus, the fix in this case is to upgrade to 0.134. However, a sci.* group doesn't normally allow binaries (does it?) and as such, is quite unlikely to have surpassed the 32-bit limit of some 4 billion plus posts, in which case this patch won't solve the problem you are seeing.
There are less than 200,000 posts to sci.electronics.design on astraweb.
Third... not so good news. There has been a longstanding but quite obscure bug in pan for quite some time, related in some way to cross- posts. The bug is obscure enough not to affect most folks, and I'd be unsure it was even there, but for the fact that I have it myself! Unfortunately, it's obscure enough and complicated enough nobody has been able to trace down the problem, and it remains unfixed.Some users of sci.electronics.design do inexplicably routinely crosspost text only messages to alt.binaries.schematics.electronics, another group I follow, so this may be the crosspost related bug.What I know about the problem so far is that it is somehow triggered by crossposts to multiple groups. I /think/ it only happens when one group involved in the cross-post gets ahead of another, perhaps you don't visit or update the behind group for some days, or perhaps you're just adding it as a new subscription after seeing sufficient cross-post activity to become interested in the other group (my case here). Somehow, something about the cross-posts gets pan mixed up, with the result being, apparently, a case of the first possibility above -- pan somehow records way high message sequence numbers for the one group, perhaps recording the numbers for the first group cross-posted to, to the second instead of using the lower numbers of the second group. As such, it then believes any new messages it sees are in fact years old and refuses to download them!If you are seeing this rather obscure behavior, I'm sad for you, but glad, in that we now have another data-point to compare against mine, and maybe some progress can finally be made on this thing!Two questions should help prove this the case or not:1) Does the problem group get cross-posts from another group you regularly visit, or may have visited in the past? (If no such cross-posts are involved, it's unlikely you're experiencing this same bug.)
2) If you start a clean pan instance (say by moving the ~/.pan2 directory elsewhere, when pan's not running of course, so it has to start a new one), setup the server, and ONLY visit the problem group, WITHOUT visiting anything else first, so the ONLY data it has is from that problem group, the group should load just fine. However, start trying to bring the saved data from the other groups back in and the problem group may go unresponsive again. The existing posts will remain (until they expire anyway), but pan will refuse to update the group any further. (This is what I see here. Again, if with a completely clean pan instance, with the problem group the FIRST you visit so no history or possibility at all for the cross-posted messages to interfere since the other group hasn't been visited yet, if it does NOT load just fine, you're seeing a different problem.)I got exactly these results, but strangely the problem re-occurs when I load other groups not including the cross-post suspect alt.binaries.schematics.electronics. I am not sure if the problem would re-occur in a few uses even with no other groups visited yet since I added new groups as soon as it appeared that sci.electronics.design was working, I will test that later, after taking care of some other work.
Thanks, Glen
| [Prev in Thread] | Current Thread | [Next in Thread] |