[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Pan-users] Re: Pan and rsync
[Pan-users] Re: Pan and rsync
Fri, 6 Aug 2010 09:35:08 +0000 (UTC)
Pan/0.133 (House of Butterflies; GIT a971f44 branch-testing)
Joe Zeff posted on Thu, 05 Aug 2010 22:09:00 -0400 as excerpted:
> On 08/05/2010 04:49 PM, Duncan wrote:
>> That could be it.
> And then again...
> As I started to read each group, Pan marked all of the old posts read,
> leaving me with nothing but the newest ones. Interesting. Last time,
> when I didn't check the newsrc file, it didn't do that, just expected me
> to read them. Now, it's marking them read as I enter the group. OK, I
> can live with that.
(Referring to the files as I type, I didn't have all this memorized, tho
Charles might! =:^)
Yeah. The (per-server) newsrc file(s) simply lists per-group xref
sequence numbers (and whether the group is subscribed, etc). These are
the ranges of posts it has seen and marked read, but they /only/ track
subscribed and marked-read.
*Most* of the the rest of the group data is in the groups subdir, in the
files named after the individual groups. These contain pan's header
cache, formatted in three parts as mentioned in the remarks at the top of
the file. The first two parts are xref and authors shorthands. These
substantially reduce string duplication and thus pan's memory usage, as
well as file-size on-disk. The third part is a per-post per-header record
of the overview data, message-id, subject, author (shorthanded if it's one
of the most frequent posters, looks like the shorthand lists are 72
entries long), references line (if present), time, xref (shorthanded), two
lines of attachment info.
The other file of interest here is newsgroups.xov (xover, the overviews
index file). This file consists of lines tracking per-group total article
count, unread count, and per-group-per-server high-water-mark, which is
the xref-sequence number of the highest visible (generally, newest) post
on the server. The difference between the stored high-water-mark and the
one pan gets when it checks with the server is the number of messages on
that server that pan believes it doesn't know anything about yet.
Keep in mind that it's only the newsrc files that are standardized. The
rest are pan-specific format.
You manually synced (when rsync failed) the newsrc files, which tracked
the messages you'd read, but apparently, the newsgroups.xov file wasn't
synced either, so when pan checked and saw a much higher high-water-mark
on the server, it requested all the messages between its (old) stored high-
water-mark and the current one it just got from the server. That's the
header download. Once it compared those headers with the records in the
newsrc files, however, which it does on entry to the group, it realized it
already knew about them and that you'd already read them, so it then
marked them read.
But it's likely a good thing you did download them. If the groups subdir
didn't sync either, as now seems likely, those headers/overviews wouldn't
be listed there either, and had the newsgroups.xov file been synced, pan
would have had not actual record of those headers as it would have
believed it had already seen them (and that they'd simply expired or you
deleted them), based on the high-water-mark from the newsgroups.xov file.
If you expire the headers anyway, no big deal, unless you wanted to
actually see them for some reason, but if you keep everything around
unexpired, as I do, that would have been a gap in the archive, as I said,
messages that from pan's viewpoint, you'd seen and then deleted.
Meanwhile, I may have figured out why you're not getting a good sync.
Near the bottom of the rsync manpage, there's a list of environmental vars
it uses. In addition to the rsync-specific ones which don't look
interesting to us, there's three more general ones, one (which it checks
two vars for) of which DOES look interesting.
According to the manpage, rsync (on the local/client side) looks for the
USER or LOGNAME var, using this as the default username sent to rsyncd.
If neither is set, it defaults to user "nobody".
I suspect that however you're running rsync, both of these (USER would be
most common) are either unset or not exported, so rsync isn't seeing
them. User "nobody" is unlikely to have permissions to write to the pan
parts of your homedir, so...
I don't know what your user umask, the default perms mask, or your group
setup. On high security systems, user's home group will be the same name
as the username, a 1:1 correspondence, and default umasks will default to
0077, meaning group and world perms default to nothing. On moderate
security systems, there'll be one or more "users" groups, with more than
one login user per group, and umasks will default to 0027 or similar,
world still gets no permissions but the group gets read/execute, just not
write. by default. Low security systems often default to 0022, both group
and world read/execute, not write.
If user "nobody" is in the same primary group as your user, the medium
security 0027 umask would give it read but not write access unless you'd
changed the default permissions, or no access if it's not in the same
group. With low security, it'd get read but not write access either way.
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman