Re: [Pan-users] pan marking all new articles as read

[Duncan]

Jim Henderson posted on Mon, 04 Jan 2016 17:19:53 +0000 as excerpted:

> On Mon, 04 Jan 2016 07:30:20 -0700, manthony-hrKqIoV4s10AvxtiuMwx3w
> wrote:
> 
>> I am using the latest version of Pan on an Ubuntu 14.04 system.  A few
>> weeks ago, it started marking all new downloads in all newsgroups as
>> "Read".  I deleted all the articles and re-loaded them, but the problem
>> persists.  Just before this problem started, there was a day when there
>> were no new articles on any newsgroups.  What do I need to learn to fix
>> this?
> 
> I've seen this on occasion, and find that resetting the newsrc-* file
> pointer for the group/groups affected resolves the issue for me.

Yes.

Technically the way it works is this:  Every message to a newsgroup has 
two different IDs, a global-scope Message-ID that's supposed to uniquely 
identify the message no matter on what server the message appears (FWIW, 
this Message-ID is identical in specification and purpose to the one 
found in email messages, as news messages inherit the email message 
definitions and are essentially the same but for a couple of headers), 
and a server-scope per-group message sequence number, as seen in the xref 
header, that increases for each message the group gets on that news 
server, but that is unique only within the context of that specific 
server for that specific group, and only as long as the server doesn't 
"reset" its message sequence number counts.

It is this second ID, the per-server per-group message sequence number 
that is in question, here.  A news client like pan can keep track of this 
number to know what messages it has already seen, and indeed, when a 
client changes groups, the news server tells it the low and high water 
marks, that is, the lowest and highest sequence numbers for which it 
currently has posts. precisely to allow a client to compare that to its 
own record of the highest number message it has seen, so the user can, if 
desired, request the messages between the highest one the client had 
previously seen, and the highest one the server now has (or all of them, 
lowest to highest, if it's the first time the user has visited the group, 
or if they haven't visited in long enough that the server has expired all 
messages that the client had previously seen).

But that all assumes the server's own idea of message sequence numbers 
never resets, and that the sequence numbers continue to increase from one 
visit to the next.

If the server itself crashes and loses its current group message sequence 
numbers, and didn't have backups with this information stored somewhere, 
then when the server admin brings it back online, its message sequence 
numbers will reset, starting at 1 for each group, once again.

What happens then when a news client connects and sees say a high water 
mark message number of 23876 for a particular group as reported by the 
server, when the client previously had seen all messages up to say 143823 
for that group, is that the client says to itself, "Oh, I've seen all 
those old messages, there's nothing new for me to download", when in 
fact, what actually happened is that the server reset its per-group 
sequence numbers and the new numbers have nothing to do with the old ones 
other than the fact that they are generally much lower, as the server  
has seen far fewer messages since the reset than it had before.

In particular, when a lot of groups at the same time suddenly have no new 
messages, when they should have many, a server message sequence number 
reset is almost always the culprit.

As it happens, pan tracks the per-server per-group message sequence 
numbers it has already seen in these newsrc files, one per configured 
server (with the servers.xml file mapping between server and 
corresponding newsrc file), with these newsrc files in turn listing each 
group, one per line, along with the message sequence numbers seen, if pan 
has visited the group.

So when a server reset like that happens, you can delete its 
corresponding newsrc file, and that should reset pan's side of things as 
well.

Alternatively, if it's for just one or a few groups (some "servers" are 
actually multiple servers behind the scenes, and will track for example 
binaries and text groups separately, so sometimes only the one set or the 
other will be reset), you can open the newsrc file in a standard text 
editor and delete the numbers for a corresponding group, manually, 
instead of deleting the whole newsrc file and thus records for all groups 
on that server, at once.

Either way, of course do this when pan isn't running, so it doesn't 
rewrite the bad numbers when you switch groups or exit.

Meanwhile, pan actually uses the message-id as the file name, in its 
message cache.  When you have multiple servers configured, that's how it 
knows a message is already in-cache from one server, if it tries to 
download the same message from another server, because it already has it 
cached by the message-ID.  Of course this works best if the cache isn't 
so small that for instance the attachment was already downloaded and 
saved, and the cached message deleted, before pan got to that message on 
the slowest server.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman