[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Pan-users] Pan/0.142 Header shown twice?
From: |
Duncan |
Subject: |
Re: [Pan-users] Pan/0.142 Header shown twice? |
Date: |
Mon, 9 Oct 2017 04:09:02 +0000 (UTC) |
User-agent: |
Pan/0.143 (Quaint little villages here and there; 54f7ff4db) |
Duncan posted on Mon, 09 Oct 2017 02:58:17 +0000 as excerpted:
> Dave posted on Sun, 08 Oct 2017 23:46:08 +0000 as excerpted:
>
>> About the most complicated thing I do these days
>> apart from a bit of bash scripting is edit the Pan source code to
>> increase the number of threads to 30 per server whenever a FreeBSD port
>> comes along.
>
> As you may know but it's worth repeating for others that may be reading
> in any case, editing source code isn't absolutely essential for that
> tweak.
> [T]he non-code workaround is to edit the config file, in this case
> servers.xml in the pan config dir (~/.pan2 by default[).]
> Meanwhile, I've one more point to make on the server connections topic,
> but I'll make it in a followup post, as it could potentially lead to its
> own subthread...
So here's that followup...
Are you sure you actually need 30 connections? In my experience, there
are only three (rather narrow) cases in which more than, say, 8-10
connections, and often, any more than 4, is actually useful, and one of
those three cases won't count here.
IOW, more connections doesn't always mean better throughput. In fact, in
practice more connections often means either worse throughput or more CPU
and memory usage to get the same thruput, due to all those extra fetch
threads trying to use the same already maxed out pipe.
The three exceptions are:
1) Users where the server caps the per-connection bandwidth but not the
total bandwidth, so the more connections you can use the more bandwidth
you get.
But providers with per-connection bandwidth caps generally allow only a
handful of connections anyway; usually in the single digits and often
only 2-4, only very rarely 20+ connections, because they're using a per-
connection bandwidth limit along with a total connections limit, as a
substitute for what they /really/ intend, a per-user bandwidth limit,
because the per-connection limit is easier or less resources intensive to
track in their server software.
Given that you mentioned a 30-connection limit, this wouldn't seem to
apply to you. 20+ connections allowed tends to be much more common where
they're encouraging you to use more bandwidth, either to encourage tier
upgrades on per-month plans, or to encourage you to eat thru your block
plan as fast as possible so you'll need to renew it.
2) Those with poor quality internet connections in general. In this
case, the individual connections will be throttled to something well
below the nominal pipe bandwidth, both that of the user's internet
connection and that allowed by the server, due to TCP's packet-loss
throttling mechanisms.
If this is you, OUCH, I feel for you! Been there, and it SUCKS! While
upping the number of news server connections may help to some degree for
news, it doesn't help for general web browsing, media streaming (which
can be simply unworkable, period, unless you can download it to cache to
watch later... which of course is something you might be using the binary
newsgroups for in place of real-time streaming), any sort of interactive
use, etc.
Due to the degree to which such poor internet situations suck in general,
most people eventually either give up or switch to something else, if
they possibly can. But 20-30 connections may indeed help somewhat for
news, in the mean time. This one's the most legit reason, but as I said,
if it applies, I sure feel for you!
And the one that won't apply in this case, but can be legit where it
does...
3) Multiple different users and/or devices connecting to the same NSP
account at once. A good example I know is the guy who had pan
downloading big binaries (TV programs, many from overseas) to his media
server with some of his connections, a couple others used by pan on his
personal computer for text groups, and a few others used by his wife/gf
on her machine, keeping up with her embroidery or whatever it was (cross-
stitch, some such...) groups.
Obviously this makes good use of an NSP's 20+ connection limit, but
doesn't require a whole lot of connections per individual pan instance.
In fact, the GNKSA-max four connections per server may well be plenty,
even for that binary-downloader media machine, if neither of the first
two apply as well.
Meanwhile, every connection has its own overhead, TCP-connection related,
CPU-thread context switches, and multiple write threads accessing storage
at the same time thereby heavily fragmenting all files compared to fewer
download threads at once. (Of course this latter cost will apply mostly
to conventional "spinning-rust", not so much to SSDs, which should handle
up to gigabit pipes at least, with ease.) If you can get the same thruput
with fewer connections, it'll mean lower CPU usage, less disk thrashing
if you're still on spinning rust, and less connection overhead as well.
So fewer connections tends to be better, as long as you're not throttling
thruput, which you shouldn't be under normal circumstances.
In fact, here, at least some years ago, I found I could max my local
internet pipe connection with only two connections! Of course my local
internet pipe is bigger now (I just upgraded to 100 megabit, there's 300
available but I can't justify it yet, and gigabit available in some areas
of town but not yet to me), I'm on ssds so the storage IO shouldn't be a
bottleneck, and my middle-aged AMD bulldozer-1 fx6100 6-core moderately
overclocked to 4 GHz is better than back then as well, but I'd still be
extremely surprised if 2-4 connections couldn't max things out just as
well as 8 or 20 or 50, and above say 8 connections, I'd expect CPU-
overhead on the 6-core to have an increasing effect.
So what I'm saying is if you've not actually checked and can justify
double-digit download connections, you may well want to do some checking,
because it's actually quite likely 30 connections is doing you more harm
than good. OTOH if you're running a 16-core-plus monster with a gigabit
internet pipe and storing to ssds, then perhaps all those threads /do/
increase your thruput, as they might if you have a really poor internet
connection and are seeing tcp-throttling due to lost packets on
individual connections. But other than that, if you haven't checked, I'd
suggest doing so. I certainly changed my mind when I saw two connections
maxing out my pipe just as well as many more did. (Tho I did see a
reasonable boost with two over just one.)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman