Re: [Gnu-arch-users] New feature at the mirror + request for help

[Andrew Suffield]

On Mon, Mar 29, 2004 at 11:53:01AM -0500, Evan Powers wrote:
> Now, obviously BitTorrent has its own scalability limits. If you are 
> interested, I suggest you read
> 
> http://bitconjurer.org/BitTorrent/BitTorrentecon.pdf
> 
> Basically, the current bottleneck appears to be the bandwidth overhead of the 
> tracker (quite small relative to the bandwidth used for uploading pieces of 
> the file), which hasn't yet been reached. The largest known deployments have 
> had over 1000 simultaneous downloaders.

This is not by coincidence. For example, we kicked about the notion of
seriously distributing Debian daily updates via bittorrent one
evening. We fairly rapidly concluded that it just wasn't possible;
there are hundreds of servers that receive the daily mirror pulse, and
a server that can handle a bittorrent tracker for that scale of
bandwidth just does not exist. (This is on the scale of "When potato
was released, the resulting bandwidth consumption demolished the UK
academic network").

I expect that most of the other people who work on a similar scale
have had similar discussions and reached the same
conclusion. Bittorrent hasn't visibly reached the limits because
they're easy to predict and not waste time trying it in the scenarios
where it won't work. This isn't very visible because most people
aren't going to encounter those scenarios.

We also worked out roughly how to solve these problems, but it's
moderately complicated and frankly everybody had better things to do
than pursue the issue. Debian does not suffer from a shortage of
distribution bandwidth; our existing infrastructure will cope for the
forseeable future.

Bittorrent *doesn't* scale because you can only really have one
server. All that it does in practical terms is give you a healthy
multiplier for the capacity of that server. In its current form, it is
no use for scenarios where you need a lot of servers. This problem is
not hugely difficult to solve, but it's not trivial either and it
hasn't been solved. (In fact it's a classical wide-area clustering
problem, so the form of the solution is well understood).

-- 
  .''`.  ** Debian GNU/Linux ** | Andrew Suffield
 : :' :  http://www.debian.org/ |
 `. `'                          |
   `-             -><-          |

signature.asc
Description: Digital signature