[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Pan-users] Re: Reproducible crash retrieving headers in million-header
From: |
Duncan |
Subject: |
[Pan-users] Re: Reproducible crash retrieving headers in million-header group |
Date: |
Fri, 30 Jul 2010 09:59:12 +0000 (UTC) |
User-agent: |
Pan/0.133 (House of Butterflies; GIT a971f44 branch-testing) |
K. Haley posted on Fri, 30 Jul 2010 00:43:04 -0600 as excerpted:
> On 7/29/2010 10:51 AM, Wayne E. Nail wrote:
>> If this log is useless please tell me, and I will run Pan again under
>> the same conditions but with whatever modifications you advise.
>>
>> I am pretty sure I have had Pan download *all* headers from a
>> well-over-one-million-headers newsgroup in the past.
>>
> According to the log Pan is crashing while trying to allocate a mere
> 5KB. This implied that it's hitting a limit to the amount of memeory it
> can allocate. I know other users regularly view such large groups so
> I'm fairly certain it's not a bug. At this point all I can suggest is
> checking how much memory Pan is using when it crashes.
Also, as the user that runs pan, check your resource limits. You can do
that from bash using its ulimit builtin, as "ulimit -a" (-a = all, give
you the whole list. In particular, you're interested in max memory size
(-m), and max virtual memory (-v).
There's actually two limits per setting, a hard limit and a soft limit.
By default, you get (or set) the soft limit. You can also get/set the
hard limit using -H, so -Hv to get/set the hard virtual memory limit. The
soft limits are what is normally used, but you can reset your soft limit
up to the hard limit. The hard limit can be reduced but not increased,
unless you're root, tho many are unlimited by default, so unless they're
set by the system, you can choose whatever value you want the first time,
or simply leave them unlimited.
But it's good to set some sort of limit, to prevent a fork-bomb taking out
the entire system. With a limit set, it'll simply run into that limit,
and crash, instead of taking out the entire system. Thus, some
distributions set a limit of some kind, perhaps related to the amount of
memory you have, by default.
FWIW, for my regular username, with 6 gigs of RAM and 20 gigs swap[1], I
set a soft limit of a gig of memory, two gigs of virtual memory. Hard
limits are double that, 2 gigs physical RAM, 4 gigs virtual. My admin
username (the one I give full password-less sudo access, which my regular
username doesn't get, and from which I run all my Gentoo compiles) has
somewhat higher limits, IIRC 2 gigs normal RAM, as there's an app or two
that I run, that require over a gig to compile. Before I bumped it up for
them, I was running into the ulimits trying to compile those programs.
(The one I remember was kmail, from kde3. Of course I'm running kde4 now,
and it seems better in that regard, but I've kept the higher settings,
just in case.)
In addition to being able to get/set ulimits in your login shell (just
setting them in any old shell won't do any good, except for programs run
from that shell), the pam system can control resource limits using the
pam_limits.so module. Those settings should be in /etc/limits.conf or a
similar file (simply /etc/limits, here on Gentoo), and in /etc/security/
limits.d/*. See the pam_limits (8) and limits.conf (5) manpages for
information on those.
___
[1] 20 gigs of swap because: It's 4x 5-gig swap partitons, one on each of
four different drives, set to the same priority, so the kernel stripes
them for 4X the normal swap speed. I chose 5 gigs per partition as I
normally run 8 gigs RAM, but a stick died that I've not replaced, so I
only have six at present. A mainline kernel suspend image can be up to
half the size of physical RAM, so that would be 4 gig. The suspend image
uses a swap partition, but it can only use one, so I setup five gig swap
partitions so there'd be plenty of room for the 4 gig suspend image on one
of them. The other three are the same size so the partition layout of
each disk remains identical, so with four disks each with a 5 gig swap
partition, I end up with the otherwise rather insane total of 20 gigs of
swap! =:^/
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
- [Pan-users] Reproducible crash retrieving headers in million-header group, Wayne E . Nail, 2010/07/29
- Re: [Pan-users] Reproducible crash retrieving headers in million-header group, K. Haley, 2010/07/30
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group,
Duncan <=
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group, Wayne E . Nail, 2010/07/30
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group, Duncan, 2010/07/30
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group, Zing, 2010/07/30
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group, Wayne E . Nail, 2010/07/30
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group, Wayne E . Nail, 2010/07/30
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group, Duncan, 2010/07/31
- Re: [Pan-users] Re: Reproducible crash retrieving headers in million-header group, K. Haley, 2010/07/31
- [Pan-users] Re: Reproducible crash retrieving headers in million-header group, Duncan, 2010/07/31