Re: [Pan-users] New Windows installer

[Duncan]

Alan Taylor posted on Sun, 20 Mar 2016 11:13:19 -0300 as excerpted:

> On 15 March 2016 at 09:34, Steve Davies
> <address@hidden> wrote:
>> Hi,
>>
>> With all of the recent activity I thought it only fair that I should
>> assemble a new Win32 installer build. It is at the usual location:
>>
>> https://sites.google.com/site/paninstall/
>>
>> It runs for me here but is not heavily tested as I am not a big NNTP
>> user anymore. My attempts in 2014 were a bit crashy, so I've attempted
>> a more pure GIT compile this time.
>>
>> Let me know your results here.
>>
> I have installed this latest Windows version and here are my results:
> Windows 10 Pro 64 bit, version 1511, OS build 10586.164, Intel Core2 Duo
> E7500 @ 2.93GHz, 6GB ram I uninstalled a previous version of Pan using
> the Windows uninstall process.  A new install would look OK, but when I
> actually try to run the program Pan kept failing with the message :"This
> application has requested the Runtime to terminate it in an unusual way"
> 
> The fix/solution is to delete the existing C:/users/USERid/.pan2
> directory before trying to install, because (I guess) the uninstaller
> doesn't delete it and its presence somehow messes up the new install.
> After deleting that directory Pan will install and run.

It would be interesting to bisect it down to a specific file, and then, 
assuming it's a settings file, down to a specific setting in the file.  
But that's a lot of work some people won't have the time/motivation to 
do.  But speaking from experience, knowing exactly what setting triggered 
the problem feels pretty good, as it helps put you in control of the 
problem, not the other way around. =:^)

(Bisect means a repeating process of splitting the problem space more or 
less in half and testing the half to see whether it's in that half or the 
other half, then splitting the bad half again, and repeating, until you 
get down to a specific unit.  For this type of bisect, that unit would 
typically be a single file and then a single line within that file.

A different kind of bisect is the source-code commit bisect that git semi-
automates these days, where a known good version and a known bad version 
are reported, and git bisect is used to find a source commit roughly half 
way in between them to test.  That is then reported good or bad and git 
chooses the bad half and again gives you a commit roughly half way 
between, until a specific commit is found to be bad, and can be reported 
along with a description of the bug for the devs to figure out what went 
wrong.  This powerful tool allows even non-devs to do most of the work of 
finding the problem, even if they don't understand a line of code, as 
long as they have access to a git repo with the sources and can build 
them into a binary to test with.  As such, it has made it far easier for 
the user reporting a bug to contribute to actually fixing it, even when 
as I said they can't make sense of even a single line of code, and that 
in turn has been one of many factors behind git taking the development 
world by storm, since it allows even non-devs to contribute in ways most 
never thought possible, before.)

> I think I had to reboot before my ISP would recognize Pan, otherwise I
> got a "authentication required" error.  Authentication is *not*
> required on my fibre-to-the-home setup.

One of the problems Linux users have been reporting is sort of the 
opposite of that, tho I don't believe the problem was an authentication 
error.  Pan would work the first time after its directory was cleaned out 
and a new configuration created, but shutting it down and restarting with 
even the new configuration would result in problems, again.  Pan would 
only work when started fresh, and couldn't work with an existing config, 
even if it had just rebuilt it, at all.

So it's nice to see at least you aren't seeing that problem, even if a 
reboot is needed to get pan working properly.

But that's one reason why bisecting the problem with the old config may 
be useful, because it may yield further hints on what's causing pan to 
dislike any config at all, on the Linux side.

FWIW, I haven't seen that problem here, but I've been running git pan for 
years, and it may be that the incremental upgrades have prevented me from 
seeing the problem.  Either that, or I happened to miss the trigger 
commit, until something else fixed it, or perhaps it's not on the main 
branch that I'm running yet, as people have been testing some preliminary 
patches and I've lost track of whether that particular issue was with 
them or on the main branch.

Either way, seeing the reverse pattern, that pan doesn't work right at 
first, but does after a reboot, is kind of interesting.

> Then pan runs fine, until the article-cache fills up.  This is a problem
> I have had with previous versions, I don't recall how many-seems like
> all of them for the past 5(?) or more years.
> 
> I download a lot of binaries, some quite large, so even with a 100 MB
> article-cache it does fill up.  When it does fill up, instead of
> stopping, or erroring, Pan keeps on downloading, but only getting
> fragments of some of the attachments.  When I stop Pan, the
> article-cache is cleared by the program (because I have that option set)
> and then runs again OK until the article-cache fills again.  For
> example, downloading a binary with 50 MB attachments eventually fails.
> You might think it with a 100 MB cache it would fail on the third file
> but no, it runs for a while, for quite a few files, before failing.
> 
> I have increased my article-cache to 1000 MB and am running that now,
> but I think that also will eventually fill up.  I suppose the fix is
> that Pan should watch the free space in the article-cache and clear out
> unneeded items when space is short, or clear unneeded items as soon as
> they are saved to disk.  (I don't know how the article-cache is used so
> I don't know when an article is no longer needed.  Seems to me it would
> be unneeded when saved to disk.  Or if the idea is to keep as many as
> possible for re-use, then Pan could issue a warning and pause, not
> silently fail.)

How do you use pan to download those binaries?  There are two 
possibilities, one of which should work fine with the default 10 MiB 
cache size, one of which will require a far larger cache size; I've 
worked with 12 GiB caches here.  So see which of the following matches 
your download method and adjust either your method or your cache size, 
accordingly.

In the one case, which to my knowledge works fine with even the tiny 
default 10 MiB cache, you select messages before downloading anything, 
and tell pan to download and save the attachments directly, using the 
"save" functions directly, before anything is cached.  This works, or has 
in the past, anyway, because pan works on only a few messages at a time 
and knows what messages it has already saved the attachments from and can 
delete to keep the cache at its configured size.

That's apparently how pan's long-time head dev, Charles Kerr, worked, and 
pan was designed around that download method, so it has worked pretty 
well.   If that's how you're using pan and you're still having cache 
expiry problems, then you may well have found a new bug, or at least I've 
not seen it discussed here.


The other alternative, which I tend to prefer, and which I'd guess most 
folks who started on USENET with MSOE (as I did, back in the MS Windows 
95 era, with IE/OE4, before it was integrated into MS Windows 98) or 
similar tools may find more intuitive, is to first download messages of 
interest to cache (using cache article instead of save), then once 
they're in local cache, browse thru them, saving off attachments or 
deleting them as desired.

Back in 2002 or so when I first started using pan, I couldn't figure out 
why it was deleting most parts of most messages before I'd even read 
them.  It turned out that it was because I was using the second method, 
download to cache and then go thru the messages again when they're local, 
while pan's default cache size was designed with the first method in mind.

If this is your problem as it was mine, there are two possible fixes.  
One is to change the way you work to fit the way pan was designed to 
work.  Don't download to cache (the second method) any more.  Instead, 
save attachments off directly (the first method), so they're saved as 
they download and pan's tiny 10 MiB default cache works fine.

Alternatively, do what I did and continue using method two, but just 
massively (and I mean massively, into the multiple GiB) increase the 
cache size, until it's large enough to hold all the raw messages you 
download in a session, so they're not deleted before you can actually 
sort thru them and decide what you really want to keep and what you'd 
prefer to simply delete once you see more of it than the sample here or 
there that you might have downloaded to check before you started the 
massive download to cache.

This has worked fine for me since I first figured out what the problem 
was and increased the size of my cache accordingly, and continues to 
work, today.  Tho FWIW, back when I first started, pan's cache size max 
was I think 1 GiB, and I needed larger, about 4 GiB, IIRC, by my 
calculations.  I filed a bug and Charles agreed to increase the max size 
to a then massive 20 GiB, IIRC.  (This really was massive, back then, as 
many people had hard drives of only single-digit GiB, so this was several 
times the size of many people's hard drives, back then!)

I don't believe there is a particular limit on the cache size since the 
rewrite, or at least no practical limit for 64-bit pan.  32-bit pan might 
still be limited to 32 GiB or something, due to integer size limits.

Even my text-group instance of pan is set to something like a 5 GiB cache 
size, tho I've only something like 1.3 GiB actually there, because I'm 
using it as an unexpiring message archive.  I don't do binaries like I 
used to, but I was using a dedicated cache partition, some 12 GiB in 
size, for my binary pan instance for awhile, tho I didn't recreate one on 
my SSDs when I switched over.  But that worked fine.  If I get seriously 
back into binaries again, I'll probably create a new one, maybe 24 GiB 
this time.


The one problem with such large caches, however, particularly on spinning 
rust, is that pan reads all those messages into memory as it starts, 
building a threading tree.  Unfortunately, pan doesn't store that tree 
anywhere but memory, so it has to build a new one every time you restart 
pan.  That means that from a cold cache on spinning rust, pan can take 10 
minutes or more to start.  So I set it to start with my X/KDE session 
(you'd start it at boot) and generally kept it running.  Obviously 
startup time issues don't tend to be a problem with the default 10 MiB 
cache, but they certainly can be once you're talking multiple-GiB 
caches.  Tho with the SSDs, I've not noticed it except that the storage 
activity LED stays solid for a minute or so when I start KDE if I'm 
starting from cold-cache, and pan's systray icon takes a minute or so to 
show up.


So as I said above, if you're using the download to cache first and sort/
save from there method, you WILL need a larger cache, perhaps multiple 
GiBs large, as I'm using here.  But if you're using the direct-save 
method for which pan and its default 10 MiB cache size was originally 
designed, and you're still having issues, you may have found a new bug.  
Tho as I said, that 10 MiB default cache size was coded back when when 
single-digit GiB drives were common, actually likely before that as 
that's just when I started using pan, and it's possible message sizes 
have increased to the point that 100 MiB may be necessary for some, now, 
particularly if you're doing 10+ connections, even when using the direct-
save method.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman