[Pan-users] Pan doesn't delete cached article files on GUI article delete - Bug? Change in behavior?

[Duncan]

FWIW, current git pan, git commit in the headers, but from help/about, 
it's 80566c8.

Setup:

As regulars here will likely know from my previous mentions, I maintain 
three separate pan profiles or instances, taking advantage of the fact 
that if pan finds PAN_HOME set to some directory in its environment when 
it starts, it will look for its config files, cache, etc, there, instead 
of using the (Unix) default $HOME/.pan2/ for its files.

So I have a pan starter script with three symlinks, pan.text, pan.test 
and pan.bin, that (among other things) uses the name of the symlink it 
was started as to point pan at the appropriate data files for that 
profile.

pan.text is for the text groups I follow and is the only profile I use at 
all regularly.  These days, the only text "groups" I follow are those on 
news.gmane.org, which is a mailing list to newsgroup and web interface 
service and archive (see http://gmane.org for more on it), allowing me to 
follow various mailing lists including this one for pan as newsgroups, 
instead of mailing lists.  However, now years ago, when my ISP still 
hosted newsgroups, I subscribed to several of the ISP-specific newsgroups 
there, and I actually still have them archived in my pan.text instance, 
in case I want to look back at some of the discussion or more likely, 
find the email address of some of the old users, assuming of course that 
it's still valid.

Of course for both servers (the old ISP server set to a fake localhost 
address of 127.0.0.1, with connections set to zero, so it never tries to 
update and only actually updates the gmane server) I have expiration set 
to never-expire, so the headers stay around, and for my text profile, my 
cache size is set to several gigs, while last I checked, I had only 3/4 
gig of actual messages in cache, so the cached messages stay around as 
well.

The pan.test profile, meanwhile, is for the occasional fetching of some 
message someone mentions here, for troubleshooting, where I don't 
actually want to subscribe to the group in question, and for otherwise 
simply browsing groups I may or may not subscribe to, later.  But if I 
subscribe to groups in the test profile, it's only for temporary 
convenience, as the intent is to keep the various newsrc and similar 
group tracking files as well as the cache deletable, thereby keeping this 
unimportant data from clogging my other two profiles, which I then use 
only for groups I am interested enough in to subscribe to longer term, 
never visiting unsubscribed groups in them.

But I only use pan.test occasionally, either when I'm troubleshooting 
some pan problem as seen in a specific message, or for temporary 
browsing.  I have its cache configured as several gig, unexpiring, as 
I'll often download some stuff from a few groups, then spend several days 
(or longer, as I may drop it for months at a time and only come back to 
finish the job much later).

The third profile, pan.bin, is of course for my binaries subscriptions.  
I do have a ($50) 1 TB block account from astraweb, and every once in 
awhile, I'll be bored and go download some stuff.  But honestly, I only 
actually do that a few times a year, sometimes only once a year or even 
less frequently, and while that block account is over two years old now, 
IIRC, I think I may have downloaded maybe 50 GiB.  At current annual 
usage, or even doubling or tripling that, the 1 TB block should last me 
well over a decade, and may in fact be effectively a lifetime 
subscription, as I'm nearing 50 so there's a fair chance I've under 30 
years worth of newsgroup activity left, and at current rates, that's 
about when I'd finally use up the 1 TB.

This profile and servers too is set to several gigs cache, unexpiring, so 
I can download a bunch of stuff to cache, and sort thru it in the 
newsgroup at my leisure, which may well be months, even years, before I 
go fetch more.  (Of course it doesn't hurt that astraweb, like many 
dedicated news providers, has retentions going back many years now and 
possibly unexpiring, as well, as retentions are still increasing.  So 
it's not as if I'll miss out if I don't get it before it expires, as was 
the case on my old ISP server, for instance.)

Problem:

All of which leads to what I'm posting about.  I haven't actually 
downloaded anything in any profile other than text in some months, maybe 
a year, and was bored a few days ago, so decided to go look at what I 
might have left to go thru on the testing and binary profiles, and clean 
it up, possibly to download some more if there's not too much and my 
interest lasts thru actually cleaning up what I already have.

So I start my usual going thru the newsgroups, deleting messages I'm not 
interested in, saving attachments to appropriate locations if I am and 
then deleting those, etc.

Only I noticed the message delete behavior was different than I 
remembered it.

As I remember it, deleting messages would actually delete the messages 
from the message cache, as well.  So once I was done cleaning up a group 
and searched the cache for message files containing a newsgroups header 
with the group I had just cleaned up, there wouldn't be many (if any) 
messages from that group, left.  And once I was done with all groups, the 
cache would be almost empty, tho I think there were usually a few 
fragments around, often individual parts of multipart posts that I hadn't 
actually deleted in the newsgroup, hoping the rest of the parts would be 
there and I could save the whole thing when I checked again.  If I had 
actually fully cleaned up all groups, tho, I could simply delete any 
stray messages still left in the article-cache dir, leaving it clean for 
my next download event.

But now, deleting messages doesn't seem to actually delete them.  They do 
get deleted from the header pane, and pan still has them tracked as seen 
so (presumably) won't show me the deleted messages again, unless I delete 
its tracking.  But the cache stays the same size as best I can tell, and 
even after there's nothing else showing up in a group as cached, grepping 
the cache folder for that newsgroup name still give me the same number of 
hits as before I deleted the messages in pan.

So either something's different, and pan isn't deleting the actual in-
cache messages as it used to, or I'm mis-remembering, and pan never did 
actually delete the articles in the cache, and it was only my manual 
deletion of all files in the article-cache after I was thru cleaning up 
the groups before the next download session, that actually deleted 
anything from the article-cache itself.

But the trouble is, it has likely been a year, maybe more, since I 
actually downloaded anything, and while I'm sure I remember pan behaving 
as I described, deleting files from cache as I deleted the messages in 
the GUI, maybe I'm remembering old C pan's behavior, and the C++ rewrite, 
now rather old itself, has never actually worked that way at all.

Questions:

So now the question.  Has anyone else noticed this change in behavior?  
Can anyone else confirm whether older C++ pan, say 0.130 or whatever, 
deleted the message files in the cache when the headers were deleted in 
the GUI headers pane?  If C++ pan deleted files from the cache from its 
beginning, 0.90, and doesn't, now, 0.140 (actually live-git after 0.139's 
release), when did that behavior change?

Maybe it's a bug, either introduced fairly recently, likely after 0.139's 
release, or gcc5 related despite the fact that I'm building with 
-D_GLIBCXX_USE_CXX11_ABI=0, since if my memory is correct, it worked fine 
a year or whenever ago, and I'd have been using gcc4.something then.

So is anyone building pan from current git using gcc4, and does current 
git pan still delete messages from the cache when they're deleted in the 
gui when built with gcc4?  (If it matters, since I'm posting this message 
via pan, to gmane, the exact git commit I built from should be in the 
headers, but pan's help/about also lists it, as git 80566c8 .)

Indeed, I'm /almost/ sure it /was/ working a year or whenever ago, as one 
group I had visited was mostly cleaned up, only 80 cached message files 
as hits when I searched for that newsgroup name in the headers, which is 
how I know deleting the last few didn't change a thing, as it's STILL 
giving me 80 file hits.  I'm reasonably sure I had downloaded way more 
than 80 messages originally, so the rest of them must have been cleaned 
up when I deleted the obvious spam and of no particular interest to me 
messages back then.


Meanwhile, I'm aware that many users simply use the default cache size or 
use a much smaller cache, and let pan manage it automatically, often not 
downloading to cache at all, but rather, downloading and saving files 
directly, so they don't need a multi-gig cache to browse the pre-
downloaded messages from once they're all locally cached, as I generally 
do.  Obviously, these folks aren't likely to care one way or the other, 
since their cache is small enough pan rotates individual messages in and 
out multiple times over in a single download.

So am I the only one doing download to cache, and going thru messages 
(not already saved attachments) once they're downloaded, and thus the 
only one who actually cares about pan's behavior when a message is 
deleted in the GUI, because for everyone else, the message will likely be 
overwritten in cache within minutes, anyway?

If there are others who care about this, /should/ pan delete the messages 
from cache once they're deleted from the GUI?  Does it even matter to 
anyone else?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman