From MAILER-DAEMON Wed Jan 07 06:06:37 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AeBWH-0007l9-B9 for mharc-gnunet-developers@gnu.org; Wed, 07 Jan 2004 06:06:37 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AeBSe-0006J5-4Z for gnunet-developers@gnu.org; Wed, 07 Jan 2004 06:02:52 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AeBRq-0005TU-W1 for gnunet-developers@gnu.org; Wed, 07 Jan 2004 06:02:34 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AeBQL-00042z-2F for gnunet-developers@gnu.org; Wed, 07 Jan 2004 06:00:29 -0500 Received: (qmail 11541 invoked by uid 65534); 7 Jan 2004 09:58:57 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp007) with SMTP; 07 Jan 2004 10:58:57 +0100 X-Authenticated: #1791878 Date: Wed, 7 Jan 2004 10:58:55 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <8910050069.20040107105855@gmx.net> To: gnunet-developers@gnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] Some small patches X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 11:06:35 -0000 Hello, when roaming through the source trying to find the reasons for some minor annoyances I patched a few places: Index: src/applications/afs/esed2/block.c =================================================================== RCS file: /var/cvs/GNUnet/GNUnet/src/applications/afs/esed2/block.c,v retrieving revision 1.10 diff -r1.10 block.c 77c77 < if ((0 == STAT(filename, &st))) { /* if file exists, try truncate */ --- > if ((0 == STAT(filename, &st)) && getFileSize(filename) > filesize) { /* if file exists, try truncate */ This fixes a bug I encountered trying to continue a download on a mounted windows partition, when the "truncation" of a file to a bigger size failed. As gnunet grows downloaded files automatically the "truncation" in this case is not really necessary... Index: src/applications/afs/module/high_mysql.c =================================================================== RCS file: /var/cvs/GNUnet/GNUnet/src/applications/afs/module/high_mysql.c,v retrieving revision 1.11 diff -r1.11 high_mysql.c 574c574 < "REPLACE INTO data%uof%u " --- > "REPLACE DELAYED INTO data%uof%u " This speeds up insertion/indexation a little when using mysql as database. cvs server: Diffing src/applications/afs/tools Index: src/applications/afs/tools/gnunet-insert.c =================================================================== RCS file: /var/cvs/GNUnet/GNUnet/src/applications/afs/tools/gnunet-insert.c,v retrieving revision 1.85 diff -r1.85 gnunet-insert.c 38c38 < printf("%8u of %8u bytes inserted\n", --- > printf("%8u of %8u bytes inserted\r", This allows using the -V option with gnunet-insert without cluttering the screen with progress messages... As I looked through the sources I saw a few places where MALLOCs were used for temporary variables (string buffers etc.). The only reasons I can imagine for doing so are: - replacing buffer overflows on the stack by corruptions in the heap, which are harder to exploit successfully (not really a good reason) - maximum stack size is reduced during runtime (should not be that big a problem) If I haven't overlooked another justification for this, I think it would be better to replace those MALLOCs by stack allocations wherever possible. It speeds things up a little, removes possible memory leaks, and reduces heap fragmentation. Ciao, Hendrik From MAILER-DAEMON Wed Jan 07 08:49:41 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AeE45-0002KG-32 for mharc-gnunet-developers@gnu.org; Wed, 07 Jan 2004 08:49:41 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AeE42-0002IH-Hi for gnunet-developers@gnu.org; Wed, 07 Jan 2004 08:49:38 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AeDyS-00009U-3r for gnunet-developers@gnu.org; Wed, 07 Jan 2004 08:44:24 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AeDvZ-0007Jj-Bs for gnunet-developers@gnu.org; Wed, 07 Jan 2004 08:40:53 -0500 Received: (qmail 10325 invoked by uid 65534); 7 Jan 2004 12:39:20 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp015) with SMTP; 07 Jan 2004 13:39:20 +0100 X-Authenticated: #1791878 Date: Wed, 7 Jan 2004 13:39:19 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <276455441.20040107133919@gmx.net> To: gnunet-developers@gnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] Usage of super queries X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 13:49:40 -0000 Hello, watching the stats of my gnunetd (which runs for some months now, with interruptions) I never once saw a value different from 0 for "# lookup (super query)". Does this mean that my server never got asked for a super query block? If yes, what are those used for then? Shouldn't they enable more efficient downloads by bundling queries? Maybe there is a bug preventing the usage of super queries? Ciao, Hendrik From MAILER-DAEMON Wed Jan 07 15:51:35 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AeKeN-0001SF-9e for mharc-gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:51:35 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AeKeK-0001RX-Or for gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:51:32 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AeKdn-0001IP-Kf for gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:51:30 -0500 Received: from [213.243.153.14] (helo=smtp2.pp.htv.fi) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AeKdn-0001Hx-4x for gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:50:59 -0500 Received: from posti.pp.htv.fi (posti.pp.htv.fi [212.90.64.50]) by smtp2.pp.htv.fi (Postfix) with ESMTP id 482D42973A9; Wed, 7 Jan 2004 21:49:29 +0200 (EET) Received: from cs71085.pp.htv.fi (cs71085.pp.htv.fi [212.90.71.85]) by posti.pp.htv.fi (8.11.1 (Revision 1.5+JAGae91741+JAGae92668) /8.11.1) with ESMTP id i07JnST06875; Wed, 7 Jan 2004 21:49:28 +0200 (EET) Date: Wed, 7 Jan 2004 21:49:39 +0200 (EET) From: Igor Wronsky X-X-Sender: nag@localhost.localdomain To: Hendrik Pagenhardt Subject: Re: [GNUnet-developers] Some small patches In-Reply-To: <8910050069.20040107105855@gmx.net> Message-ID: References: <8910050069.20040107105855@gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Igor Wronsky List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 20:51:33 -0000 On Wed, 7 Jan 2004, Hendrik Pagenhardt wrote: > Index: src/applications/afs/module/high_mysql.c > =================================================================== > RCS file: /var/cvs/GNUnet/GNUnet/src/applications/afs/module/high_mysql.c,v > retrieving revision 1.11 > diff -r1.11 high_mysql.c > 574c574 > < "REPLACE INTO data%uof%u " > --- > > "REPLACE DELAYED INTO data%uof%u " > This speeds up insertion/indexation a little when using mysql as database. Looking at 4.0.17 reference manual, I'm not so sure if this is a good idea. Particularly, I suspect that errors might go unnoticed or that the database size might be incorrectly reported when delayed inserts are used. The manual also says that insert delayed should only be used if you're really sure you need it. All in all, I don't think gnunet is generally so faultless that we could afford adding any potential troublemakers. Igor From MAILER-DAEMON Wed Jan 07 15:54:55 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AeKgF-00024j-7w for mharc-gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:53:31 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AeKg9-0001zp-IA for gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:53:25 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AeKRY-0006Wu-O2 for gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:38:52 -0500 Received: from [213.243.153.173] (helo=smtp3.pp.htv.fi) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AeKOZ-0005HD-5a for gnunet-developers@gnu.org; Wed, 07 Jan 2004 15:35:15 -0500 Received: from posti.pp.htv.fi (posti.pp.htv.fi [212.90.64.50]) by smtp3.pp.htv.fi (Postfix) with ESMTP id 90B6627B8FC; Wed, 7 Jan 2004 21:33:41 +0200 (EET) Received: from cs71085.pp.htv.fi (cs71085.pp.htv.fi [212.90.71.85]) by posti.pp.htv.fi (8.11.1 (Revision 1.5+JAGae91741+JAGae92668) /8.11.1) with ESMTP id i07JX8T25853; Wed, 7 Jan 2004 21:33:14 +0200 (EET) Date: Wed, 7 Jan 2004 21:32:02 +0200 (EET) From: Igor Wronsky X-X-Sender: nag@localhost.localdomain To: Hendrik Pagenhardt Subject: Re: [GNUnet-developers] Usage of super queries In-Reply-To: <276455441.20040107133919@gmx.net> Message-ID: References: <276455441.20040107133919@gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Igor Wronsky List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 20:53:29 -0000 On Wed, 7 Jan 2004, Hendrik Pagenhardt wrote: > watching the stats of my gnunetd (which runs for some months now, with > interruptions) I never once saw a value different from 0 for "# lookup > (super query)". Does this mean that my server never got asked for a > super query block? If yes, what are those used for then? Shouldn't they > enable more efficient downloads by bundling queries? Maybe there is a > bug preventing the usage of super queries? Nope. That is a misleading debug statement I once put there as I tracked data flow inside gnunetd. The value should never be nonzero as super-queries are not something that should be looked up from disk. I'll replace that particular entry with a more proper, incrementing counter. Igor From MAILER-DAEMON Thu Jan 08 04:14:58 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AeWFm-0004Gd-7x for mharc-gnunet-developers@gnu.org; Thu, 08 Jan 2004 04:14:58 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AeWFk-0004FM-2a for gnunet-developers@gnu.org; Thu, 08 Jan 2004 04:14:56 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AeWFD-00040n-5Z for gnunet-developers@gnu.org; Thu, 08 Jan 2004 04:14:54 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AeWFC-0003zd-AT for gnunet-developers@gnu.org; Thu, 08 Jan 2004 04:14:22 -0500 Received: (qmail 19856 invoked by uid 65534); 8 Jan 2004 08:12:51 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp013) with SMTP; 08 Jan 2004 09:12:51 +0100 X-Authenticated: #1791878 Date: Thu, 8 Jan 2004 09:12:50 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <1569163674.20040108091250@gmx.net> To: Igor Wronsky Subject: Re[2]: [GNUnet-developers] Some small patches In-Reply-To: References: <8910050069.20040107105855@gmx.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 09:14:56 -0000 >> [proposal of delayed inserts] > Looking at 4.0.17 reference manual, I'm not so sure if this is a good > idea. Particularly, I suspect that errors might go unnoticed or that > the database size might be incorrectly reported when delayed inserts > are used. The manual also says that insert delayed should only be used > if you're really sure you need it. I'm not sure how many more uncertainties delayed inserts would introduce. I tested it on my machine and it worked really well. I even ran gnunet-check -a after a big insertion session (not really to check this, but because I suspected inconsistencies from a not gnunetd related crash of my machine) without any errors showing up. So I think the use of delayed inserts could be made an option for the more dangerous living folks among us... > All in all, I don't think gnunet is generally so faultless that we > could afford adding any potential troublemakers. You're probably right at this stage of development. I thought a bit about the topic, and I think a good way to increase insertion throughput might be the bundling of inserts within gnunet. This probably could even be more efficient than the delayed inserts. The abysmal performance of inserts is IMHO closely related to the sequential nature of the insertion process (correct me if I'm wrong). And it's not helping that we can't profit from the potentially parallel select and insert capabilities of the database, because every bucket is locked with a semaphore when a request is in progress. BTW which threads can run in parallel when gnunetd is running? I would hope that at least one thread for each connection (local or remote) is used? For mysql this might be improved by collecting the inserts per bucket in a separate thread and when a threshold number or a timeout is reached from the last insert then a REPLACE statement with multiple value tuples can be created and sent to the mysql server. Of course this would skew the table quotas and of course this is more difficult to handle if errors occur, but I think it might be worth the hassle because one of the more annoying "features" of GNUnet is the slow insertion/indexing process. Which of course might lead to less acceptance among users and hindering the willingness to publish content... Ciao, Hendrik From MAILER-DAEMON Thu Jan 08 16:39:15 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Aehs3-0004JG-26 for mharc-gnunet-developers@gnu.org; Thu, 08 Jan 2004 16:39:15 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Aehs0-0004Im-Q6 for gnunet-developers@gnu.org; Thu, 08 Jan 2004 16:39:12 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AehrT-0004CK-QP for gnunet-developers@gnu.org; Thu, 08 Jan 2004 16:39:10 -0500 Received: from [128.10.2.6] (helo=newman.cs.purdue.edu) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AehrT-0004CD-BQ for gnunet-developers@gnu.org; Thu, 08 Jan 2004 16:38:39 -0500 Received: from gecko.cs.purdue.edu (gecko.cs.purdue.edu [128.211.1.61]) by newman.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i08Kb9VN019519 for ; Thu, 8 Jan 2004 15:37:09 -0500 (EST) Received: from gecko.cs.purdue.edu (localhost [127.0.0.1]) by gecko.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i08Kb9dr015549 for ; Thu, 8 Jan 2004 15:37:09 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Christian Grothoff Organization: Secure Software Systems Lab, Computer Science, Purdue University To: gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] Some small patches Date: Thu, 8 Jan 2004 15:37:08 -0500 User-Agent: KMail/1.4.3 References: <8910050069.20040107105855@gmx.net> In-Reply-To: <8910050069.20040107105855@gmx.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200401081537.08821.grothoff@cs.purdue.edu> X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 21:39:13 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday 07 January 2004 04:58 am, Hendrik Pagenhardt wrote: > cvs server: Diffing src/applications/afs/tools > Index: src/applications/afs/tools/gnunet-insert.c > =================================================================== > RCS file: > /var/cvs/GNUnet/GNUnet/src/applications/afs/tools/gnunet-insert.c,v > retrieving revision 1.85 > diff -r1.85 gnunet-insert.c > 38c38 > < printf("%8u of %8u bytes inserted\n", > --- > > > printf("%8u of %8u bytes inserted\r", > > This allows using the -V option with gnunet-insert without cluttering > the screen with progress messages... Probably a good idea (applied). > As I looked through the sources I saw a few places where MALLOCs were > used for temporary variables (string buffers etc.). The only reasons I > can imagine for doing so are: > - replacing buffer overflows on the stack by corruptions in the heap, > which are harder to exploit successfully (not really a good reason) > - maximum stack size is reduced during runtime (should not be that big > a problem) > > If I haven't overlooked another justification for this, I think it would > be better to replace those MALLOCs by stack allocations wherever > possible. It speeds things up a little, removes possible memory leaks, > and reduces heap fragmentation. The reason is to minimize the stack size -- or better, make it predictable and small. GNUnet is using lots of threads, and a stack overflow on thread stacks has horrible consequences on some systems (segfault) -- and happens easily with a default size of 1k. Now, we specify a larger stack when we create a thread, but given GNUnet's sometimes rather long call-sequences, it is quite difficult to get a good estimate -- and just saying 200k would be quite wasteful. So the policy is roughly to MALLOC everything above 64 bytes. Speed is not an issue (really, GNUnet's CPU performance is dominated by knapsack solving and then after a while public-key crypto, but you can't really measure the MALLOC cost), memory leaks are easily found with valgrind and modern malloc implementations do not really have issues with heap fragmentation, especially not considering the allocation behavior of GNUnet, so no worries there. Given the choice between debugging a spurious, rare segfault without stack-trace (corrupt!) under BSD due to a stack overflow and finding a leak with valgrind, I go for valgrind any day... Christian -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQE//b909tNtMeXQLkIRAkVlAJwKGwwW1EQVRHyJRSlQMWH9ERvwRQCeP3U9 p5XcP7bHa8BSwsbrdotV8GU= =LFVg -----END PGP SIGNATURE----- From MAILER-DAEMON Thu Jan 08 19:37:37 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Aekee-0001VC-V5 for mharc-gnunet-developers@gnu.org; Thu, 08 Jan 2004 19:37:36 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AekPS-0007g3-U8 for gnunet-developers@gnu.org; Thu, 08 Jan 2004 19:21:55 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Aehkj-0003Av-0A for gnunet-developers@gnu.org; Thu, 08 Jan 2004 16:32:12 -0500 Received: from [128.10.2.6] (helo=newman.cs.purdue.edu) by monty-python.gnu.org with esmtp (Exim 4.24) id 1Aehhu-0002os-76 for gnunet-developers@gnu.org; Thu, 08 Jan 2004 16:28:46 -0500 Received: from gecko.cs.purdue.edu (gecko.cs.purdue.edu [128.211.1.61]) by newman.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i08KREVN019366 for ; Thu, 8 Jan 2004 15:27:15 -0500 (EST) Received: from gecko.cs.purdue.edu (localhost [127.0.0.1]) by gecko.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i08KREdr015445 for ; Thu, 8 Jan 2004 15:27:14 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Christian Grothoff Organization: Secure Software Systems Lab, Computer Science, Purdue University To: gnunet-developers@gnu.org Subject: Re: Re[2]: [GNUnet-developers] Some small patches Date: Thu, 8 Jan 2004 15:27:13 -0500 User-Agent: KMail/1.4.3 References: <8910050069.20040107105855@gmx.net> <1569163674.20040108091250@gmx.net> In-Reply-To: <1569163674.20040108091250@gmx.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200401081527.13996.grothoff@cs.purdue.edu> X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 00:37:35 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 08 January 2004 03:12 am, Hendrik Pagenhardt wrote: > I thought a bit about the topic, and I think a good way to increase > insertion throughput might be the bundling of inserts within gnunet. > This probably could even be more efficient than the delayed inserts. The > abysmal performance of inserts is IMHO closely related to the sequential > nature of the insertion process (correct me if I'm wrong). And it's not > helping that we can't profit from the potentially parallel select and > insert capabilities of the database, because every bucket is locked with > a semaphore when a request is in progress. Again, this type of optimization is likely to cause some form of trouble (like the asynchronous errors that you noted) and somehow sounds even worse than 'DELAYED' to me (but I don't know enough about MySQL to truely comprehend the extend of trouble DELAYED may or may not cause, so I'll leave that decision to Igor). Furthermore, I am not sure that insertion speed would be so much of an issue once we have the insertion/download-manager (far far in the future) where all of these things would just go into the background. And even now, why is it a problem to run 'gnunet-insert' overnight (assuming your machine is on 24/7)? > BTW which threads can run in > parallel when gnunetd is running? I would hope that at least one thread > for each connection (local or remote) is used? We never used a thread per peer-connection (2 total, always) and since 0.6.1 we only use one thread for all local clients (before one thread per client). The rules for all of these threads are that they must not block (more than for a bounded amount of disk-IO), and I don't see anything wrong with that. Note that there are other threads for background jobs that take longer, block indefinitely and for reading from the sockets. Just the actual processing of p2p messages is done by 2 threads. > For mysql this might be improved by collecting the inserts per bucket in > a separate thread and when a threshold number or a timeout is reached > from the last insert then a REPLACE statement with multiple value tuples > can be created and sent to the mysql server. Of course this would skew > the table quotas and of course this is more difficult to handle if > errors occur, but I think it might be worth the hassle because one of > the more annoying "features" of GNUnet is the slow insertion/indexing > process. Which of course might lead to less acceptance among users and > hindering the willingness to publish content... So what would be wrong with just backgrounding the insert process? (Not that you could not do 'nohup gnunet-insert XXX &' already, but some people definitely would want a GUI for that and others would want to make sure that the process is resumed after a reboot, so some more code would definitely be nice there -- but not critical right now IMO). C -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQE//b0h9tNtMeXQLkIRAmQqAJsG2elcHvDrGDJbssiqAgDU/jmikACfQBEE E6rnCUpla8r0eqLIwwu9zX8= =sPEW -----END PGP SIGNATURE----- From MAILER-DAEMON Fri Jan 09 03:51:44 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AesMp-0005JP-Fh for mharc-gnunet-developers@gnu.org; Fri, 09 Jan 2004 03:51:43 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AesMl-0005I4-S8 for gnunet-developers@gnu.org; Fri, 09 Jan 2004 03:51:39 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AesME-00059u-Rs for gnunet-developers@gnu.org; Fri, 09 Jan 2004 03:51:37 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AesME-000598-6j for gnunet-developers@gnu.org; Fri, 09 Jan 2004 03:51:06 -0500 Received: (qmail 19112 invoked by uid 65534); 9 Jan 2004 07:49:34 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp021) with SMTP; 09 Jan 2004 08:49:34 +0100 X-Authenticated: #1791878 Date: Fri, 9 Jan 2004 08:49:33 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <1333518968.20040109084933@gmx.net> To: Christian Grothoff Subject: Re[2]: [GNUnet-developers] Some small patches In-Reply-To: <200401081537.08821.grothoff@cs.purdue.edu> References: <8910050069.20040107105855@gmx.net> <200401081537.08821.grothoff@cs.purdue.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 08:51:41 -0000 >> This allows using the -V option with gnunet-insert without cluttering >> the screen with progress messages... > Probably a good idea (applied). And while you are at it ;-) : Index: src/applications/afs/tools/gnunet-download.c =================================================================== RCS file: /var/cvs/GNUnet/GNUnet/src/applications/afs/tools/gnunet-download.c,v retrieving revision 1.41 diff -r1.41 gnunet-download.c 140c140 < printf("Download at %8u out of %8u bytes (%8.3f kbps)\n", --- > printf("Download at %8u out of %8u bytes (%8.3f kbps)\r", > Given the choice between debugging a spurious, rare segfault without > stack-trace (corrupt!) under BSD due to a stack overflow and finding a > leak with valgrind, I go for valgrind any day... I understand that now. I never wrote programs were stack size was that much of an issue (always using defaults like 1M or more). I still think that the code would look much cleaner with stack based allocations, but when the priorities are platform multiplicity and performance is not that big an issue in this area, I'll can live with that ;-) Ciao, Hendrik From MAILER-DAEMON Fri Jan 09 14:10:00 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Af21A-0007vJ-2N for mharc-gnunet-developers@gnu.org; Fri, 09 Jan 2004 14:10:00 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Af217-0007uP-4F for gnunet-developers@gnu.org; Fri, 09 Jan 2004 14:09:57 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Af20Z-0007nm-S4 for gnunet-developers@gnu.org; Fri, 09 Jan 2004 14:09:54 -0500 Received: from [213.243.153.14] (helo=smtp2.pp.htv.fi) by monty-python.gnu.org with esmtp (Exim 4.24) id 1Af1CA-0004Zh-W3 for gnunet-developers@gnu.org; Fri, 09 Jan 2004 13:17:19 -0500 Received: from posti.pp.htv.fi (posti.pp.htv.fi [212.90.64.50]) by smtp2.pp.htv.fi (Postfix) with ESMTP id 69BF0296DB0; Fri, 9 Jan 2004 19:15:48 +0200 (EET) Received: from cs71085.pp.htv.fi (cs71085.pp.htv.fi [212.90.71.85]) by posti.pp.htv.fi (8.11.1 (Revision 1.5+JAGae91741+JAGae92668) /8.11.1) with ESMTP id i09HFlT21896; Fri, 9 Jan 2004 19:15:47 +0200 (EET) Date: Fri, 9 Jan 2004 19:16:02 +0200 (EET) From: Igor Wronsky X-X-Sender: nag@localhost.localdomain To: Hendrik Pagenhardt Subject: Re[2]: [GNUnet-developers] Some small patches In-Reply-To: <1569163674.20040108091250@gmx.net> Message-ID: References: <8910050069.20040107105855@gmx.net> <1569163674.20040108091250@gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Igor Wronsky List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 19:09:58 -0000 On Thu, 8 Jan 2004, Hendrik Pagenhardt wrote: > I'm not sure how many more uncertainties delayed inserts would > introduce. I tested it on my machine and it worked really well. I even > ran gnunet-check -a after a big insertion session (not really to check > this, but because I suspected inconsistencies from a not gnunetd related > crash of my machine) without any errors showing up. I'm not saying it would cause anything in normal operation. What I'm saying is that if something goes wrong by some reason in some system, we might not get to know that in the delayed setup and might even go on believing that everything is proceeding fine. > So I think the use > of delayed inserts could be made an option for the more dangerous living > folks among us... We might make it selectable by a 'wizzard' .conf option but thats about it at the moment. > nature of the insertion process (correct me if I'm wrong). And it's not > helping that we can't profit from the potentially parallel select and > insert capabilities of the database, because every bucket is locked with > a semaphore when a request is in progress. I don't actually know how necessary that locking is. I initially put it there to be certain of being on the safe side. > For mysql this might be improved by collecting the inserts per bucket in > a separate thread and when a threshold number or a timeout is reached > from the last insert then a REPLACE statement with multiple value tuples > can be created and sent to the mysql server. > but I think it might be worth the hassle because one of > the more annoying "features" of GNUnet is the slow insertion/indexing > process. Which of course might lead to less acceptance among users and > hindering the willingness to publish content... For one thing, I'm not going to implement such a thing, and if someone else does, fine, but he'll make sure its working *pretty damn well* before the current module gets replaced by it. ;) The fact is that with my setup I can probably index so much stuff in a day that I wouldn't be able to download the same amount in a month through GNUnet. If these figures ever get more balanced - for example by some really clever routing modifications - I might reconsider. Igor From MAILER-DAEMON Fri Jan 09 17:55:06 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Af5X0-0000GI-CP for mharc-gnunet-developers@gnu.org; Fri, 09 Jan 2004 17:55:06 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Af5Ww-0000Er-Jo for gnunet-developers@gnu.org; Fri, 09 Jan 2004 17:55:02 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Af4wJ-0007ih-4s for gnunet-developers@gnu.org; Fri, 09 Jan 2004 17:17:42 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AexRr-00031s-Ia for gnunet-developers@gnu.org; Fri, 09 Jan 2004 09:17:15 -0500 Received: (qmail 9483 invoked by uid 65534); 9 Jan 2004 13:15:45 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp021) with SMTP; 09 Jan 2004 14:15:45 +0100 X-Authenticated: #1791878 Date: Fri, 9 Jan 2004 14:15:43 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <11030577.20040109141543@gmx.net> To: gnunet-developers@gnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] MYSQL module X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 22:55:03 -0000 Hello, always searching for easy hacks to improve performance of GNUnet, I looked at the source of "estimateAvailableBlocks" in the MYSQL module. What I really want is to reduce the number of calls to this function (I'll try that at the weekend), but for now I have a question about the specific implementation. First I noticed, that the average row length is multiplied with the number of rows (costing two SQL statements), when MYSQL returns the size of the data file in the "Data_length" column of SHOW TABLE STATUS. So row counting should not be necessary. Secondly this calculation omits the space the table indexes use. As those currently add another 20% of the table data size to the space consumed by the content storage, I would suggest to either document that the quota is used for the data files only, or, preferably, to adjust the calculation to include the index size. That could be easily done by adding the "Data_length" and "Index_length" columns of SHOW TABLE STATUS, still saving the row count. A way to shrink the indexes, and possibly speed up data access at a whole, might be to not use a primary key on the bucket tables. It could be replaced by an index over the first 3 bytes of the hash. Randomness assumed, this would be sufficient to distinctly identify 256^3 rows, which is an order of magnitude higher than what could be placed in a bucket. I tested this on my system with a copied GNUnet table (125MB data file, 25 MB index) and the index shrank to less than half (9 MB). Query performance was as fast or even faster this way. And the reduced index size should help caching. The only drawback is, that the "writeContent" method must be rewritten to prevent duplicates (f.i. by first trying to UPDATE the row, and if this results in 0 affected rows to then insert it). And of course the table definitions must be changed. Which could be done via some ALTER TABLE statements during init when required. Ciao, Hendrik From MAILER-DAEMON Fri Jan 09 20:04:50 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Af7YX-0004HF-L5 for mharc-gnunet-developers@gnu.org; Fri, 09 Jan 2004 20:04:49 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Af7YT-0004G3-Lq for gnunet-developers@gnu.org; Fri, 09 Jan 2004 20:04:45 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Af6O6-00053j-Pb for gnunet-developers@gnu.org; Fri, 09 Jan 2004 18:50:30 -0500 Received: from [212.90.64.119] (helo=smtp1.pp.htv.fi) by monty-python.gnu.org with esmtp (Exim 4.24) id 1Af6Nl-0004y4-AE for gnunet-developers@gnu.org; Fri, 09 Jan 2004 18:49:37 -0500 Received: from posti.pp.htv.fi (posti.pp.htv.fi [212.90.64.50]) by smtp1.pp.htv.fi (Postfix) with ESMTP id 7CA83804A8; Sat, 10 Jan 2004 00:48:02 +0200 (EET) Received: from cs71085.pp.htv.fi (cs71085.pp.htv.fi [212.90.71.85]) by posti.pp.htv.fi (8.11.1 (Revision 1.5+JAGae91741+JAGae92668) /8.11.1) with ESMTP id i09Mm1T10064; Sat, 10 Jan 2004 00:48:01 +0200 (EET) Date: Sat, 10 Jan 2004 00:48:16 +0200 (EET) From: Igor Wronsky X-X-Sender: nag@localhost.localdomain To: Hendrik Pagenhardt Subject: Re: [GNUnet-developers] MYSQL module In-Reply-To: <11030577.20040109141543@gmx.net> Message-ID: References: <11030577.20040109141543@gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Igor Wronsky List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jan 2004 01:04:46 -0000 On Fri, 9 Jan 2004, Hendrik Pagenhardt wrote: > First I noticed, that the average row length is multiplied with the > number of rows (costing two SQL statements), when MYSQL returns the size > of the data file in the "Data_length" column of SHOW TABLE STATUS. So > row counting should not be necessary. Secondly this calculation omits > the space the table indexes use. As those currently add another 20% of > the table data size to the space consumed by the content storage, I > would suggest to either document that the quota is used for the data > files only, or, preferably, to adjust the calculation to include the > index size. That could be easily done by adding the "Data_length" and > "Index_length" columns of SHOW TABLE STATUS, still saving the row count. No. See the note before estimateAvailableBlocks(). As those two fields do not update in real time, relying on them can result in gnunetd trashing hundreds of megabytes from the database if it gets full. And that did happen before. > A way to shrink the indexes, and possibly speed up data access at a > whole, might be to not use a primary key on the bucket tables. It could > be replaced by an index over the first 3 bytes of the hash. Randomness > assumed, this would be sufficient to distinctly identify 256^3 rows, > which is an order of magnitude higher than what could be placed in a > bucket. I tested this on my system with a copied GNUnet table (125MB > data file, 25 MB index) and the index shrank to less than half (9 MB). > Query performance was as fast or even faster this way. And the reduced > index size should help caching. > The only drawback is, that the "writeContent" method must be rewritten > to prevent duplicates (f.i. by first trying to UPDATE the row, and if > this results in 0 affected rows to then insert it). And of course the > table definitions must be changed. Which could be done via some ALTER > TABLE statements during init when required. A question before dwelving further into optimization hacks, is the database really a bottleneck on your system? That is, can you afford so much upstream bandwidth that mysql can't deliver blocks fast enough? If thats not the case, there is not much point in optimizing further, imo, unless you really wish to save a few cpu cycles or a few blocks from the disk. If you really wish to make gnunet more efficient, you might look into the routing/trust business and suggest enhancements for that. I suspect thats the place where extra efficiency is called for most at the moment. Igor From MAILER-DAEMON Sun Jan 11 09:08:51 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AfgGp-0007hr-4k for mharc-gnunet-developers@gnu.org; Sun, 11 Jan 2004 09:08:51 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AfgGk-0007d1-6B for gnunet-developers@gnu.org; Sun, 11 Jan 2004 09:08:46 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AfgGA-00078A-4m for gnunet-developers@gnu.org; Sun, 11 Jan 2004 09:08:41 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AfgG9-0006fZ-6o for gnunet-developers@gnu.org; Sun, 11 Jan 2004 09:08:09 -0500 Received: (qmail 18492 invoked by uid 65534); 11 Jan 2004 13:06:28 -0000 Received: from port-ip-213-211-237-69.reverse.mdcc-fun.de (EHLO port-ip-213-211-237-69.reverse.mdcc-fun.de) (213.211.237.69) by mail.gmx.net (mp014) with SMTP; 11 Jan 2004 14:06:28 +0100 X-Authenticated: #1791878 From: Hendrik Pagenhardt (by way of Hendrik Pagenhardt ) Subject: Re: [GNUnet-developers] MYSQL module Date: Sun, 11 Jan 2004 14:06:26 +0100 User-Agent: KMail/1.5.4 To: gnunet-developers@gnu.org MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401111406.26319.Hendrik.Pagenhardt@gmx.net> X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik.Pagenhardt@gmx.net List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jan 2004 14:08:47 -0000 On Friday 09 January 2004 23:48, you wrote: > No. See the note before estimateAvailableBlocks(). As those > two fields do not update in real time, relying on them can > result in gnunetd trashing hundreds of megabytes from the > database if it gets full. And that did happen before. This is not completely correct. The "Data_length" and "Index_length" fields are correct always in regard to the size of the files on the disk. What makes it seem not reliable is the fact, that MySQL doesn't shrink the data file when rows are deleted. That only happens when OPTIMIZE TABLE is run. So to get an amount which can be used in the quota calculation one has to subtract the "Data_free" column from the sum of the other two. This is more accurate then the current solution, as the "Avg_row_length" column is just calculated from "Data_length" and the number of records in the table (including deleted rows), which when multiplied with the actual row count is nothing more than an estimate of the consumed space in the data file based on outdated row length information. BTW, running OPTIMIZE TABLE on the gnunet tables is a good idea after big insertions as it balances the indexes (and in most cases shrinks them) and removes the deleted rows from the data file. Maybe this could be made an option of gnunet-check. > A question before dwelving further into optimization hacks, > is the database really a bottleneck on your system? That is, > can you afford so much upstream bandwidth that mysql can't > deliver blocks fast enough? If thats not the case, there > is not much point in optimizing further, imo, unless > you really wish to save a few cpu cycles or a few blocks from > the disk. Well, as the upstream limit of my gnunetd is reached most of the time, I have to answer: no, probably the database is not really the bottleneck for my system. It might be different on other setups, though. What I can't stand is wasting ressources. And as GNUnet is not the only application running on the system (as it is probably the norm for most current and future users) every feasible reduction in ressource usage should be implemented once the implementation of the concerned module is reasonably stable. And what I suggested reduces the number of SQL-queries done every time a block is inserted into the database by the number of buckets (in my case 24). And when I come around to implement the reduction of calls to estimateAvailableBlocks, this is further reduced by number of buckets - 1. Admitted, the queries affected are quite fast, but nevertheless they don't need to be executed and the overhead of inserting a block will be reduced by quite some margin. > If you really wish to make gnunet more efficient, you might > look into the routing/trust business and suggest enhancements > for that. I suspect thats the place where extra efficiency > is called for most at the moment. If you can point me to a place in the source were it makes sense to start looking into, I'll try my best ;-). But I'm afraid, that I'm better in making optimizations on a local scale, than on the larger design issues. Ciao, Hendrik From MAILER-DAEMON Sun Jan 11 11:11:24 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AfiBQ-0000cx-EF for mharc-gnunet-developers@gnu.org; Sun, 11 Jan 2004 11:11:24 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AfiBN-0000ce-QT for gnunet-developers@gnu.org; Sun, 11 Jan 2004 11:11:21 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AfiAr-0000GZ-TS for gnunet-developers@gnu.org; Sun, 11 Jan 2004 11:11:20 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AfiAr-0000GA-Fv for gnunet-developers@gnu.org; Sun, 11 Jan 2004 11:10:49 -0500 Received: (qmail 17912 invoked by uid 65534); 11 Jan 2004 15:02:39 -0000 Received: from port-ip-213-211-237-69.reverse.mdcc-fun.de (EHLO port-ip-213-211-237-69.reverse.mdcc-fun.de) (213.211.237.69) by mail.gmx.net (mp015) with SMTP; 11 Jan 2004 16:02:39 +0100 X-Authenticated: #1791878 From: Hendrik Pagenhardt To: Igor Wronsky Subject: Re: [GNUnet-developers] Usage of super queries Date: Sun, 11 Jan 2004 16:02:36 +0100 User-Agent: KMail/1.5.4 References: <276455441.20040107133919@gmx.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401111602.37756.Hendrik.Pagenhardt@gmx.net> Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik.Pagenhardt@gmx.net List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jan 2004 16:11:22 -0000 On Wednesday 07 January 2004 20:32, Igor Wronsky wrote: > On Wed, 7 Jan 2004, Hendrik Pagenhardt wrote: > > watching the stats of my gnunetd (which runs for some months now, > > with interruptions) I never once saw a value different from 0 for "# > > lookup (super query)". Does this mean that my server never got asked > > for a super query block? If yes, what are those used for then? > > Shouldn't they enable more efficient downloads by bundling queries? > > Maybe there is a bug preventing the usage of super queries? > > Nope. That is a misleading debug statement I once put there > as I tracked data flow inside gnunetd. The value should never > be nonzero as super-queries are not something that should > be looked up from disk. > > I'll replace that particular entry with a more proper, > incrementing counter. When you removed the statistics, you probably removed LOOKUP_TYPE_SUPER from to many places. An insertion now fails with: Jan 11 15:59:39 WARNING: server could not perform insertion Jan 11 15:59:39 WARNING: child->insert failed on level 2, pos 0, aborting! Jan 11 15:59:39 WARNING: child->insert failed on level 3, pos 0, aborting! Jan 11 15:59:39 WARNING: child->insert failed on level 4, pos 0, aborting! The gnunetd log shows: Jan 11 15:59:39 WARNING: tried to insert super query into database Jan 11 15:59:39 WARNING: unexpected super query block (case 2) This comes from manager.c "insertContent". Obviously the super queries do have to be inserted into the database after all... Hendrik From MAILER-DAEMON Sun Jan 11 13:08:20 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Afk0Z-0000Le-JY for mharc-gnunet-developers@gnu.org; Sun, 11 Jan 2004 13:08:19 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Afk0I-0000Gg-7F for gnunet-developers@gnu.org; Sun, 11 Jan 2004 13:08:02 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AfjzQ-000057-B5 for gnunet-developers@gnu.org; Sun, 11 Jan 2004 13:07:39 -0500 Received: from [213.243.153.14] (helo=smtp2.pp.htv.fi) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AfjzP-0008WU-Rf for gnunet-developers@gnu.org; Sun, 11 Jan 2004 13:07:07 -0500 Received: from posti.pp.htv.fi (posti.pp.htv.fi [212.90.64.50]) by smtp2.pp.htv.fi (Postfix) with ESMTP id 397C3297386; Sun, 11 Jan 2004 19:05:37 +0200 (EET) Received: from cs71085.pp.htv.fi (cs71085.pp.htv.fi [212.90.71.85]) by posti.pp.htv.fi (8.11.1 (Revision 1.5+JAGae91741+JAGae92668) /8.11.1) with ESMTP id i0BH5aT26301; Sun, 11 Jan 2004 19:05:36 +0200 (EET) Date: Sun, 11 Jan 2004 19:05:54 +0200 (EET) From: Igor Wronsky X-X-Sender: nag@localhost.localdomain To: Hendrik Pagenhardt Subject: Re: [GNUnet-developers] Usage of super queries In-Reply-To: <200401111602.37756.Hendrik.Pagenhardt@gmx.net> Message-ID: References: <276455441.20040107133919@gmx.net> <200401111602.37756.Hendrik.Pagenhardt@gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Igor Wronsky List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jan 2004 18:08:18 -0000 On Sun, 11 Jan 2004, Hendrik Pagenhardt wrote: > When you removed the statistics, you probably removed LOOKUP_TYPE_SUPER > from to many places. An insertion now fails with: > This comes from manager.c "insertContent". Obviously the super queries do > have to be inserted into the database after all... Yes, perfectly obvious, especially in the light of the previous discussion: they've never read from the db, so why insert at all? But they actually are, by gnunet-check, and I had forgotten that. Maybe it=A0might be better to reconstruct the superbloomfilter from scratch (since we do have the indexed files I suppose) so it would more accurately reflect the current situation. But I fixed my screw-up - thanks for reporting. Igor From MAILER-DAEMON Mon Jan 12 03:03:14 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Afx2X-0005Bb-UD for mharc-gnunet-developers@gnu.org; Mon, 12 Jan 2004 03:03:13 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Afx2U-0005Ao-Vb for gnunet-developers@gnu.org; Mon, 12 Jan 2004 03:03:10 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Afx1y-00051n-KZ for gnunet-developers@gnu.org; Mon, 12 Jan 2004 03:03:09 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1Afx1y-00051W-1l for gnunet-developers@gnu.org; Mon, 12 Jan 2004 03:02:38 -0500 Received: (qmail 15874 invoked by uid 65534); 12 Jan 2004 06:54:27 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp009) with SMTP; 12 Jan 2004 07:54:27 +0100 X-Authenticated: #1791878 Date: Mon, 12 Jan 2004 07:54:26 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <91516060.20040112075426@gmx.net> To: Igor Wronsky Subject: Re[2]: [GNUnet-developers] Usage of super queries In-Reply-To: References: <276455441.20040107133919@gmx.net> <200401111602.37756.Hendrik.Pagenhardt@gmx.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2004 08:03:12 -0000 > But I fixed my screw-up - thanks for reporting. In line 638 of manager.c is a typo now, preventing compilation... Ciao, Hendrik From MAILER-DAEMON Mon Jan 12 09:57:40 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Ag3Vc-0004rS-17 for mharc-gnunet-developers@gnu.org; Mon, 12 Jan 2004 09:57:40 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Ag3VZ-0004q7-He for gnunet-developers@gnu.org; Mon, 12 Jan 2004 09:57:37 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Ag3PA-0002tT-OL for gnunet-developers@gnu.org; Mon, 12 Jan 2004 09:51:31 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1Ag3PA-0002rB-0r for gnunet-developers@gnu.org; Mon, 12 Jan 2004 09:51:00 -0500 Received: (qmail 13600 invoked by uid 65534); 12 Jan 2004 13:49:18 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp005) with SMTP; 12 Jan 2004 14:49:18 +0100 X-Authenticated: #1791878 Date: Mon, 12 Jan 2004 14:49:17 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <1195060969.20040112144917@gmx.net> To: gnunet-developers@gnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] Patch submission X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2004 14:57:38 -0000 Hello, I hacked a small patch to reduce the number of calls to "estimateAvailableBlocks". It works by remembering the returned values for all buckets and just refreshing those made dirty when needed. I compiled and tested it on my tree and at least it doesn't make things worse. Maybe a core developer can have a look at it and comment? Ciao, Hendrik Index: src/applications/afs/module/manager.c =================================================================== RCS file: /var/cvs/GNUnet/GNUnet/src/applications/afs/module/manager.c,v retrieving revision 1.17 diff -r1.17 manager.c 27a28,29 > #include > 51a54,55 > #define DB_DIRTY_AVAILABLE MININT > 307c311,313 < = MALLOC(dbAPI->buckets * sizeof(HighDBHandle*)); --- > = MALLOC(dbAPI->buckets * sizeof(HighDBHandle)); > dbAPI->dbAvailableBlocks > = MALLOC(dbAPI->buckets * sizeof(int)); 315a322,323 > dbAPI->dbAvailableBlocks[i] > = DB_DIRTY_AVAILABLE; /* not yet initialized */ 443a452,473 > * calculates the global available space using > * cached bucket availability estimates > **/ > > int estimateGlobalAvailableBlocks() { > unsigned int i; > int ret = 0; > int perBucketQuota = getConfigurationInt("AFS", > "DISKQUOTA") * 1024 / dbAPI->buckets; > > for (i = 0; i < dbAPI->buckets; ++i) { > if (dbAPI->dbAvailableBlocks[i] == DB_DIRTY_AVAILABLE) { > dbAPI->dbAvailableBlocks[i] > = dbAPI->estimateAvailableBlocks(dbAPI->dbHandles[i], > perBucketQuota); > } > ret += dbAPI->dbAvailableBlocks[i]; > } > return ret; > } > > /** 486,490c516,517 < delta = 0; < for (i=0;ibuckets;i++) < delta += dbAPI->estimateAvailableBlocks(dbAPI->dbHandles[i], < getConfigurationInt("AFS", < "DISKQUOTA")*1024/dbAPI->buckets); --- > delta = estimateGlobalAvailableBlocks(); > 496a524,525 > dbAPI->dbAvailableBlocks[i] > = DB_DIRTY_AVAILABLE; 672,673c701,704 < dbAPI->unlinkFromDB(computeHighDB(query), < query); --- > if (OK == dbAPI->unlinkFromDB(computeHighDB(query), query)) { > dbAPI->dbAvailableBlocks[computeBucketGlobal(query)] > = DB_DIRTY_AVAILABLE; > } 722a754,755 > dbAPI->dbAvailableBlocks[computeBucketGlobal(query)] > = DB_DIRTY_AVAILABLE; 742a776,777 > dbAPI->dbAvailableBlocks[computeBucketGlobal(query)] > = DB_DIRTY_AVAILABLE; 793a829,830 > dbAPI->dbAvailableBlocks[computeBucketGlobal(query)] > = DB_DIRTY_AVAILABLE; 822d858 < unsigned int i; 869,874c905 < avail = 0; < for (i=0;ibuckets;i++) < avail += dbAPI->estimateAvailableBlocks < (dbAPI->dbHandles[i], < getConfigurationInt("AFS", < "DISKQUOTA")*1024/dbAPI->buckets); --- > avail = estimateGlobalAvailableBlocks(); 884a916,917 > dbAPI->dbAvailableBlocks[computeBucketGlobal(&query)] > = DB_DIRTY_AVAILABLE; 938a972,973 > dbAPI->dbAvailableBlocks[computeBucketGlobal(&query)] > = DB_DIRTY_AVAILABLE; 1012,1017c1047,1049 < unsigned int i; < int delta = 0; < for (i=0;ibuckets;i++) < delta += dbAPI->estimateAvailableBlocks(dbAPI->dbHandles[i], < getConfigurationInt("AFS", < "DISKQUOTA")*1024/dbAPI->buckets); --- > dbAPI->dbAvailableBlocks[computeBucketGlobal(query)] > = DB_DIRTY_AVAILABLE; > int delta = estimateGlobalAvailableBlocks(); Index: src/applications/afs/module/manager.h =================================================================== RCS file: /var/cvs/GNUnet/GNUnet/src/applications/afs/module/manager.h,v retrieving revision 1.4 diff -r1.4 manager.h 207a208,212 > * cache estimated available blocks for each bucket > **/ > int * dbAvailableBlocks; > > /** From MAILER-DAEMON Mon Jan 12 13:08:34 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Ag6UK-0008Q5-Jz for mharc-gnunet-developers@gnu.org; Mon, 12 Jan 2004 13:08:32 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Ag6U5-0008NK-CM for gnunet-developers@gnu.org; Mon, 12 Jan 2004 13:08:17 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Ag6TY-0008DX-Fm for gnunet-developers@gnu.org; Mon, 12 Jan 2004 13:08:16 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1Ag6TX-0008CM-8r for gnunet-developers@gnu.org; Mon, 12 Jan 2004 13:07:43 -0500 Received: (qmail 9878 invoked by uid 65534); 12 Jan 2004 16:59:30 -0000 Received: from port-ip-213-211-237-69.reverse.mdcc-fun.de (EHLO port-ip-213-211-237-69.reverse.mdcc-fun.de) (213.211.237.69) by mail.gmx.net (mp009) with SMTP; 12 Jan 2004 17:59:30 +0100 X-Authenticated: #1791878 From: Hendrik Pagenhardt To: gnunet-developers@gnu.org Date: Mon, 12 Jan 2004 17:59:27 +0100 User-Agent: KMail/1.5.4 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401121759.27018.Hendrik.Pagenhardt@gmx.net> Subject: [GNUnet-developers] Content Migration X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik.Pagenhardt@gmx.net List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2004 18:08:18 -0000 Hello, from looking at the source of "insertContent" in manager.c I get the impression that it is possible to overwrite an indexed CHK block with a migrated one. If the priority is big enough to jump over the migration threshold the hash is looked up and as an indexed block has an empty content field it is not recognized as a duplicate. So the index pointer block will be overwritten and becomes a "normal" CHK block. I think that's a bad thing, as the block takes more space then. I'd suggest adding a test after reading the old content to check whether it's an index block and then set the duplicate output param accordingly. Hendrik From MAILER-DAEMON Wed Jan 14 07:54:11 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AgkXC-0006Nx-Jd for mharc-gnunet-developers@gnu.org; Wed, 14 Jan 2004 07:54:10 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AgkMt-0005Cn-BY for gnunet-developers@gnu.org; Wed, 14 Jan 2004 07:43:31 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AgkJ2-0004gE-AK for gnunet-developers@gnu.org; Wed, 14 Jan 2004 07:40:03 -0500 Received: from [203.59.3.35] (helo=mail.iinet.net.au) by monty-python.gnu.org with smtp (Exim 4.24) id 1AgkBW-0003xT-Cb for gnunet-developers@gnu.org; Wed, 14 Jan 2004 07:31:46 -0500 Received: (qmail 8739 invoked from network); 14 Jan 2004 12:31:17 -0000 Received: from unknown (HELO localhost) (203.217.36.158) by mail.iinet.net.au with SMTP; 14 Jan 2004 12:31:16 -0000 Date: Wed, 14 Jan 2004 23:31:42 +1100 From: Glenn McGrath To: gnunet-developers@gnu.org Message-Id: <20040114233142.52766acb.bug1@iinet.net.au> X-Mailer: Sylpheed version 0.9.8 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="pgp-sha1"; boundary="Signature=_Wed__14_Jan_2004_23_31_42_+1100_gMH7vj1Hkh9qV/Ib" Subject: [GNUnet-developers] stale acinclude.m4 X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2004 12:54:09 -0000 --Signature=_Wed__14_Jan_2004_23_31_42_+1100_gMH7vj1Hkh9qV/Ib Content-Type: multipart/mixed; boundary="Multipart=_Wed__14_Jan_2004_23_31_42_+1100_AdAVYITPw8A2Nx97" --Multipart=_Wed__14_Jan_2004_23_31_42_+1100_AdAVYITPw8A2Nx97 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: 7bit Hi, im preparing a new debian package. I had previously had problems with the gcrypt m4 macro's included in GNUnets acinclude.m4, it called libgcrypt-config with bad options. There is a newer macro shipped with libgcrypt that works for me. I found that the acinclude.m4 has a lot of macro's that arent need, i rebuilt the acinclude.m4 with only macros that were needed and it was only 4.8KB vs 46kB for the GUNnet version. Ive only tested a few different configure options so there may still be problems, but at a minimum i think the libgcrypt macro should be updated. Ill do some more testing tommorow. Glenn --Multipart=_Wed__14_Jan_2004_23_31_42_+1100_AdAVYITPw8A2Nx97 Content-Type: application/octet-stream; name="acinclude.m4.gz" Content-Disposition: attachment; filename="acinclude.m4.gz" Content-Transfer-Encoding: base64 H4sICE8xBUACA2FjaW5jbHVkZS5tNAC9WGtz2kYU/Wx+xQ3WFJGRAbnth8alHRlkm4ZXBXbGo1JZ RguoESuqFXHcJP+9d6UFPaHOdFrsAevu2fs4e/fs4lOYEPIGVmG4edNsLufzxpJuG36wbK49/tg8 b7VaZ63vm2u2bKnqd2pjFa69itaxuvrV7VA2Ozd65601nt4YutadKRWzcoKDsbXfu5Q34SogtqOI T2uO7yFRKicIPBHTrM54fNXXriftlK3fLZp6l5P2mSdcVU5mCneC4QaT6zhkb3gtuwuY++uN65EA 2Haz8YOQwZmYVMcJzP5AkpBVafdnFcdKzPu51TjY1LjHTIZvZXzk8U9OXTr3tg6BHwWwsfoJ7XF2 J9nC5ZYS/dQvEoQowNAnt/2p/EwYT7LIzdm+7iJJpWOcrUNRqF9PGV/GXZzXMfbS9GVwiY+qCJtm MabxEI+7Ao4xuceUc3mYTJYZztOZGxWEHggmKH0hp2lKj3N6hNQ0pwVSBasHad2XcZTYBHWA2hJu 0+QVuS0f3XFbiKcbxsiQzXt/i/Q9s5CswWVA/XBHpkuXsFuvV7M4qfgjeo/eDlGIzOE4/lZSkqYN rOvh7VCfWpNJ37rTjUlvNJwpYFYAEDXWpjfW2Bhdy3y4MxpeKeBvCGXMU4A3AQAueUhYCNWP0g5T hTY+Ur96AeGKUARFzpIKMSQlIQTkz60bELDhyQ/e89pcykLb8+zQ9Sn4CxhhLPQKfgDLefC8CXlE 4jES+cQsrLX9hx9YH0jAcEr7YZ8DCBN8ht8icPSS7Lk1/2BtAn9pae/eQu3TJnBpCJJ5PvtSO449 u2qk8CriH5IsXPryLL4iieM5nOdymAf+/57Dt3k4Iw7UWNM0f2+d/TCb4UErcoybrjfUrdvhr7ej qd6Nmmqg/TIyFJAKi4lNuFv+AbfDXWyf1Y+66w0Td+lVSbvj9pe66xijvbsUwRl3aM+4W7jRPnOo B9jqYyGQ1vR+rGMQmX8qcKPd6dZQG+j1CNhZkfl7eFoR3DIBbonweUMcsgDy0WUoojZ1INYsHDvF AZcSkM759nNDAeJ+Unu7JDReWgBMyMv2AumQ1F3IPSEdDRHWndaX8aK0WfIuEBBLUpUIZQol7owG 415fl81dalxWrMno1ujooEbQRJtZ6HjuI5fmnP2ZNXkAhkNCg3B4SzkNUuyEN52N8sLcv4i/kCUV lZvbEV6SYxuVu3yA+lBPVj6l9VIJOsbtha4MwiUPg6UEL9NUsnSuqIrZjbhxuCsbHmaSata4wAs3 NUdkhO2DFZU0kD7s9rRhZBY0Ezdql35vOu3rYty6GU2mXDIve9dpU9Qe8Ty56DXujKRI7GnGrPg0 R2k+WCDn7p1mDOVoAuwnXIDN2HbNVf3RXQKhjmtTwliqRCg0Yg5V3oNzK4YpIg0TCkPtLX1P/ack 0VN+9+fV8R7b2IG9bqwEh4yXBJf3U90aGV3dgLWNpTTSNWY7vLRfk0iZYREqaecYsoBXqYCfP+Pj fq2ix8yCJmfSo7/cMsFlnCdLvHLzggc6hV4Ijs9XC0lARSZ7XUGtcEQThn6qQXi74D3jv646VfSr dip8UiG/7CT9UiyuuNSILjN7bhh6ZLfJD2zgHZp3t2iZXIenuDC4qK5tl4Jch08ZBEDzNWh4lXki EAfmhCZ1/AxwFfhruLGDR5f59JtJSIhHGgCvmzlHW4qnSM6WDwbg+bivvIuCfb6yA5ibsTyCzGH1 WR72BbZ507bhIQVq3oznSgjytpHzCGegzqCNE+oF3zMlZzm0NP+M4yv7ElAGk6y30JhISotCbxa7 YHZQ6NN9EmefaZNE54tCrKhKYZXSBwEXn5WP4VY22/WO2OCPzyHvI4fgXWspkttfftNRc0L/VSGT Ji2NlzmL+r3J1BroA92QM5cWcWehmB1ZEzyi8TaCJwEN7TgkxePOQxv2+wTv/Q8LHw/1GipQ5ORB re2ySuZDhOFTuZ1PVrhiPbRq4PNoTy4jjdRlZ5fapW5E/5mh9pq08dLAZ7al80rkrt0q8PIvXhV+ baI8RYlHuUDJzX4f4jnE34Ukmv4uFOeixhdFx6ckui3+DXSSIVifEgAA --Multipart=_Wed__14_Jan_2004_23_31_42_+1100_AdAVYITPw8A2Nx97 Content-Type: application/octet-stream; name="configure.ac.diff" Content-Disposition: attachment; filename="configure.ac.diff" Content-Transfer-Encoding: base64 LS0tIC4vR05VbmV0LTAuNi4xYS5uZXcvY29uZmlndXJlLmFjICAgIDIwMDQtMDEtMTQgMjI6Mjc6 NDAuMDAwMDAwMDAwICsxMTAwKysrIC4vR05VbmV0LTAuNi4xYS9jb25maWd1cmUuYWMgICAgICAg IDIwMDQtMDEtMDEgMDk6MDc6MzguMDAwMDAwMDAwICsxMTAwQEAgLTQ0NCw4ICs0NDQsOCBAQAog ICAgICAgd2l0aG91dF9ndGs9ZmFsc2UKICAgICAgIExERkxBR1M9Ii1MJHdpdGhfZ3RrL2xpYiAk TERGTEFHUyIKICAgICAgIENQUEZMQUdTPSItSSR3aXRoX2d0ay9pbmNsdWRlICRDUFBGTEFHUyIK LSAgICAgIEFNX1BBVEhfR0xJQigxLjIuMCwsQUNfTVNHX0VSUk9SKENvdWxkIG5vdCBmaW5kIGds aWIpLGd0aHJlYWQpCi0gICAgICBBTV9QQVRIX0dUSygxLjIuMCwsQUNfTVNHX0VSUk9SKENvdWxk IG5vdCBmaW5kIGd0ayksKQorICAgICAgQU1fR05VTkVUX1BBVEhfR0xJQigxLjIuMCwsQUNfTVNH X0VSUk9SKENvdWxkIG5vdCBmaW5kIGdsaWIpLGd0aHJlYWQpCisgICAgICBBTV9HTlVORVRfUEFU SF9HVEsoMS4yLjAsLEFDX01TR19FUlJPUihDb3VsZCBub3QgZmluZCBndGspLCkKICAgICAgZWxz ZQogICAgICAgd2l0aG91dF9ndGs9dHJ1ZQogICAgICBmaV0sCkBAIC00NTMsOCArNDUzLDggQEAK ICAgICAgd2l0aG91dF9ndGs9ZmFsc2UKICAgICAgaWYgdGVzdCAiJGJ1aWxkX29zIiAhPSAiY3ln d2luIgogICAgICB0aGVuCi0gICAgICBBTV9QQVRIX0dMSUIoMS4yLjAsLFt3aXRob3V0X2d0az10 cnVlOyBBQ19NU0dfUkVTVUxUKGdsaWIgbm90IGZvdW5kKV0sZ3RocmVhZCkKLSAgICAgIEFNX1BB VEhfR1RLKDEuMi4wLCxbd2l0aG91dF9ndGs9dHJ1ZTsgQUNfTVNHX1JFU1VMVChndGsgbm90IGZv dW5kKV0sKV0KKyAgICAgIEFNX0dOVU5FVF9QQVRIX0dMSUIoMS4yLjAsLFt3aXRob3V0X2d0az10 cnVlOyBBQ19NU0dfUkVTVUxUKGdsaWIgbm90IGZvdW5kKV0sZ3RocmVhZCkKKyAgICAgIEFNX0dO VU5FVF9QQVRIX0dUSygxLjIuMCwsW3dpdGhvdXRfZ3RrPXRydWU7IEFDX01TR19SRVNVTFQoZ3Rr IG5vdCBmb3VuZCldLCldCiAgICAgIGVsc2UKICAgICAgIEFDX01TR19DSEVDS0lORyhmb3IgZ3Rr LXdpbjMyLTIuMCkKICAgICAgIEFDX0NIRUNLX0xJQihndGstd2luMzItMi4wLCBndGtfd2luZG93 X25ldywgQUNfTVNHX1JFU1VMVCh5ZXMpLCBBQ19NU0dfRVJST1IoQ291bGQgbm90IGZpbmQgZ3Rr LXdpbjMyLTIuMCkpCg== --Multipart=_Wed__14_Jan_2004_23_31_42_+1100_AdAVYITPw8A2Nx97-- --Signature=_Wed__14_Jan_2004_23_31_42_+1100_gMH7vj1Hkh9qV/Ib Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFABTa2WWZyfXiLlL8RAjLPAJ9g7sQ1NYsMoVy9q1sCQqyZJD/PCwCaAhrU 1stAyjcrOdfLFyf2dWYtAN4= =ILrO -----END PGP SIGNATURE----- --Signature=_Wed__14_Jan_2004_23_31_42_+1100_gMH7vj1Hkh9qV/Ib-- From MAILER-DAEMON Thu Jan 15 17:23:11 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhFtO-0007hR-C9 for mharc-gnunet-developers@gnu.org; Thu, 15 Jan 2004 17:23:10 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhFtK-0007gH-Ne for gnunet-developers@gnu.org; Thu, 15 Jan 2004 17:23:06 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhFiD-00051a-Qy for gnunet-developers@gnu.org; Thu, 15 Jan 2004 17:12:09 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1Ah8Tu-0002hr-6m for gnunet-developers@gnu.org; Thu, 15 Jan 2004 09:28:22 -0500 Received: (qmail 28091 invoked by uid 65534); 15 Jan 2004 14:28:15 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp014) with SMTP; 15 Jan 2004 15:28:15 +0100 X-Authenticated: #1791878 Date: Thu, 15 Jan 2004 15:28:14 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <010462025.20040115152814@gmx.net> To: gnunet-developers@gnu.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------7ED1501242EF7D" Subject: [GNUnet-developers] Patch for network load measurement X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2004 22:23:08 -0000 ------------7ED1501242EF7D Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hello, I had some ideas to enhance the network load estimation/interpolation. The current code is much too pessimistic and this leads to sub optimal bandwidth usage by GNUnet (at least on by system). My patch changes the way intermediate loads are calculated by using a weighted average. Especially for the advanced method this has lead on my system to much higher bandwidth utilization (without shooting over the maximum values, peaks excluded). I also got rid of the "fast rise, slow falloff" post processing of the network load values, as the new method uses a more accurate moving average and the formerly implemented falloff delay was not time independent anyway... I also cleaned a few bits up (unused variables, initialization, overflowing of the system traffic counters) and added a new configuration parameter which controls the averaging period. It should be added to the [LOAD] section of gnunet.conf (gnunet.root): -------- # How long (in seconds) is the averaging period used for calculating # the current network load? When basic bandwidth limitation is used the # minimum value is 1. For advanced mode it is 2 because of the overhead # inflicted by getting the traffic values from the operating system # The default is 5 seconds. NETAVGINTERVAL = 5 -------- This time a context diff is provided to help evaluating the changes by the core developers. Ciao, Hendrik ------------7ED1501242EF7D Content-Type: application/x-gzip; name="statuscalls.c.diff.gz" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="statuscalls.c.diff.gz" H4sICBqeBkACA3N0YXR1c2NhbGxzLmMuZGlmZgDtXHlz4jgW/5t8CsFUdwMxAZsjECYzlU6ys9nN 0ZVjtrdmp1IOFuCKsRnbJJ2dynff9yTZko0BJyHdvTWhugPY0pP0jt87JHPkWvTLDgn8QX0W2k49 CM1wFgxMxwm2Bhu7L39tnO9fkKHt0B1SvzP9+uAuqP9yeuXSMHrLHlu72/Bp6Nv0znZHxIe3wPZc om/pjdaGZQ+HpDYgNZ99J8lZV6vVBQsq6G1yQAfEaDSaxDB2jN5Os0NqDXgVOOVarbak7z9MF/u2 iN7aabR39B7viyOqLzaDdk/rdAj7SsgPFh3aLiWfzs/2r08PL68PDn8lpfrU9wZ15IFF70rQDP6F D1MKjWFJ/mwQkj83ioSQmRvYI5daxPGAGfyPGYTXtttfet+bhX2g+UhOaXjv+bcXsKCgzwfCxdkD MhibPsyS2G5I/aE5oJ9CPyC75PTq+LhP6lXyyWO3SOiRcEyJa04o8YaEmoOx7KSRsRkQdzY5iq4E hLoovoBU64yrnB/wqfZyfixgw3e6+izt2G5r3QbXDmQOftUj5kRz8z33OmRLAZaE9oReTWFqpNEn OLP7MXWJZVvknsIy6OCWtSTY7mccdQmdA+/efQ4lWDCZBfSjGdiDExqOPQuo/PvwAsggnbF3j3ya UDOY+ZSEvjkcQi+gsBlRkBKCqezdjRjD7kwH6LSPj0mVTfbicP/s9OCC8d+OGgBh84765kgS9uA7 n98GGx+uD24RLdQ5q7IH+kwxYOxrnwYzB9SBTfyp0ut2tV5TSq/b03qdlPROZiH9IoCJfU4qXmJa I8e7MZ1LvqyPcIdS9xOYQl8yLtE+eAhCOhHtj2FFyxoPPZ8COMy1VmeDgrVdO7RNx/4vta7RAs5Y k6zVG3pHMwxdrt/Qe5rR3I44gK9dEtIg3PfcoT2a+TAIiBVY6o7KpeOzvYOShu0KhQJvXfq4d3G0 f3x0cnR5dPoLuynvgYKVKrg6klIadgmHGtHkSNAgHmaTUyoByOz9+svR6eXh+a97xxFBe0jKaY1m Kl0BpOHksUlKWX8kurxP5lVZ74t7j+z9kVAnoMsJGksJGimCGcwg1d2E8TA5TMwvoAto7h8/XZBY NMv4JThfOtn7DCw7OPvXKXS9PLvkPMtUh3ZXMzqKORjbLc3oNqU6pI0uAlh2c5Hybwloh8aNPC0B 7XlT5E2mhSQprmiWIJdtQ/P0lrWLCeJS0O6ux6ZrORQZg9Z67XimdT2bxlLCNn9nTcqld+h0IA7x 7sHJiw4EO4D6TisomaVULdCATNk1G02tKcMUeE/BvSBzFQDwMiu5mpZRVXE44Vhc714DDTenAbUu wWf0N2qo5egrpqwfv2B5sxuHwrX9me8DvvIpsz8nV5eHn6+Pz/b/WX6vIGaFe3UA6KMhwSgMAxwa BOCJIQ7TSUAHnmsFxBx5Gt4fxB6C9YPXDVcSuD7akm5jyP0b706AXcy1m5ZlozEArojeI9PGmEr0 28JZTDx0bOrw8RgsTNSivgENOdW7Ebl5ACysQ3NiBuziDLmSmscW91qcq8jG8ntgbKUvWKTwF1QD 7pBaMjLgvEJsUZv+qGJCRURPJBIMUCqDnCrlMhdOBSkCYJwzM4VoYxM5WSAMiaM2K62wQurYi3fS txpAJR5AnVs9JqnOUQg9xszlE14NCXVBIfYnpExyz4ZE0+EYprJmV2o3b5ATnSQ1NagDgXJ7eOQN eFQfmQouW0wuYoScLsf4qynAdF+xpqvTRfYEadXMdwUj9UYDJFSVYzEoeUTXX+NhFccFcgl6e4MA ILR3QjEsBs0NbItCzOy5zgMZuTNYlTSZfJCC/ik/qCCWzcMKXn0DlvUDC8rmSdASi2c5uLAUhMML t8t8Fm27KrqsDV4WT3pldKLMZh3IIlIzRcnzoEsUg8yBiyCXghfFWBRokUxIg4sIIJH+8xFGGVRi DAcYDBqbBkQiMgdeUyTyMiTgdoaWj1nllPo25AcMFCBngGzZHo1DakWWGWHE0PcmrCszwqjXHBgI FIh6Z4NBOLaDiAR8Yhkvji1nRmNIUsHApfeI1tTBIsvrBBbJ9EMCAMoKFAAVq5zOc2rqcBVQiuUB ODOtzcQUqyRH/FEloHTpckKFUavP5V7VuUSKKC41ttJVOh9rPbyptpbp6Ysrcw9BE5i4Oq6ozsmi nrSC4nz8IulnsCofl+tJjchiW86Y6AmYklpGCki+j0jlDXReCDrLgo7Xhh3bfQbqYFiyHtARrnad sKPGAMVVtZG8uAMRx9NhB2fyMuBh8V827iQ5l7+q9Bzg4QqagJ7M6so2xjTNqLqyAFn2rDvTHVBL AZdoe+Rk7/M12yI5Pjo9JG3camJGhbsWaGW/JRr8vnAzyP8CtqmREN+WNMJtNWyF75w5rDC821C+ DN3ZhJPIwL9Qxb8f7CFu3MDMrj6za387Oj4EceEmD6sGWvSuz2O/bldr9nqpCvq34tMKBq1mDH5F 0Bw6AGy8kl7M4S/y8StTz7rbWktvx5uN1IWJbjzBCUUbP7BctgenuCMjmQ1bnvsBc/DIAdghXAef opEb8ObgaPiODXM1zMXG2bHtDnw6geg/WOwbitI3hE/xDSH3DUYKfBPOAfOPA+qEZl+5COK+pJPp ifmFJTfFuECfJKxXE2QZ23z6B+TWQ88nFlJldYItSNpR6NWo6pReToKQMmQ8OQGMOYAwvVaWiGIS qo6YBY5iSFjCDfhzcgMixOAIMTy076iG/t+95cEEVpjFfp7FYwymH/Hq0O0PRCaJrp/3wLgjSxXY zdADpdlSKLgPWOrAGA3iMzP4GUOPMSwBtTIYezMHiDn35kMgNwDF1CQRb0imGPgwbYWQxSeBB3Ef LGS0tbUl5SHlDYzOKkgoWkKe5oCTqjRXYsjvkB43libxDDFbjabWMuJs+YkWv3eDoTdyMcD9bFE8 4jHzhO2CcXOOgr5YA4QtR3xPBp3zQWsgw0EWivoPbBt3MPBmoC9gOnFBjOsHNhJbKDFsYGvfAqoQ tOfBjVeLKWM/UFlLeKnQS0YzKwOviMBCCst3q+KQVHZPrX9VLLuYEznCWvLssHa5VWT5xZbRACuR u1uSbE1XqXLHzi0Qa8qh8n2Rc1a9MrQcelPqltXjLBop+bgxx8zV6GitpjDXzJm2wIO3e9FMC5bJ XQFGMNVKEPqDsV9GQ9LIh50PjI2szeZmv4CfUaeDYGC6wzJe1qKiZOmd48zIu2qw+D80KEXt34tA 8T2LFCukuMs3pgl6luHA8QJaVhfOJwK2Bx7gix2WS4fn52fnO8BlE01WnpAhbEEML94F/3HFPjNJ nP/hxB5JAScjRLK5y2MyYcvqJvJvLOL6PVLrfoHJqbCiGdmNgjzGNbgFLMTPNzDnWyGttqG1OnE4 +jWFUQ1SsmAy0L+JDKIXLocL4cdVQpDQSaIYIw6EwUVHmM7wn5/dUVsrMTMeKlLupXypMOUnq4hK cQ26kmnIXV1r9YzYkMFbeZMJOLYY+jBiLGPqQGx0PeqxI7i0uRmVlyLWs6MSqAWgePBPKF7iwNpv 9u8VwnkP84x0jzcsZRl86gNqHVkCASTWvynXP3VN+VVvFtj84BfiPSigULbnmfu6bL2na+1GfHTp u5aXChIkAyVeXUxviPAcROh1QMN6MgjJEE4cv4tYdj4zSZTNIKwSS03ngKuzv8LKohdZVvRiTnbB 8QBTFGgSdffQvKUBqpgM/E3Hic8wJervQMUOyAQr3Jibgpe7ceiEFYM8V0TVAR5X9anMXqT28mPG Ohh0qyEPn+Vgd3QGVmrmAJLagOU/LGdlTlOtrAs9RUOQOoqlHqnvqaOV/FigLDDgNdU4RJuGVMkc Z9iipnnOpxWTRqJcm7MldpaW5WoBnZq+GdJlGVq1TvLNmbUq7CJ2COXNUUBevTxU6lSGJgaora4m i6IJ5l0L8yFSX5CtxQxcuJMfL2FhvZtPNWdV+3GNte2cZv692XjmoxZ6S2sbrZx19vjcwF+syt5u tbV2u5Ozyv5qXFrOnvB7rbG3jS5wb/svUWPPOPSRp8I+mz67vp4m+4IKe+ohEjmsmF7eCjs/WZG/ xC7PPbwV2LMK7Ow4saIiX7W+njhqMldKVI/gcqjsNLT2dvetvJ67vP7Cc2LrKa7LI2PPra4rFJ5V XseDUd+ivh6Bz9qq6wmTyPSHHcizOj21tr4q2syovtdSCQlJVd+LL62+CyC+5LaBzZhF4QPJ0XOp XUNrd3vqw2vPWMj8xF9x2pny6Pa0TkP/q+8gFOMdhDAuBoV5ikHiYeHCqkZkN4pgFxcVOw2QhdF5 hQ2E5XsHKVGE32YDoShquriKME+VkJ0m/ZPHVnnqg4WswmBBiQcKsS0+Sfps4mtQgSzz7BhNrdNs ve0LPG1f4FsacaurdTqt73lnIGOHQBFY+BV3Bt6MPWHszW2t01LOHTzyEjP7FRTUGqyOoVCxaMaC gij2XlysLmbuDUCYlt4ZCJ+yMyCT1ufvC0DoL8qFzGw6sPae8vD3Wtb+/1mmTz30mfP58Khx+I0K 9SIH2kWDXl+dXlJVM6swX6E+Sqtep1IvjWhJnT5cXqdXn3p5XOezL6utLvsHSppatxtnZ6jdIxqK 3wU5BltRHp5TLABHOKfKzGSSrtaIsbNM9/H+T1HXSlTXkaQwyURN/WNmD26dB+LbWPfwyNgejQHl 0XLjcqXQ7zjJQnng2ybZFj/egmPUu9xF95oNrdeM4+yvsMqa3qhXRf9q/astNkvEvQYsXtdTP90w z4PEM0uLuFCUXIgrmBH8yR+FYciZrCDU9PgpiRT7+LgL1eTPF7Hu8ama0trWeu1W6unSlzMr15rT SvP1lv4/kSKK9d5NAAA= ------------7ED1501242EF7D-- From MAILER-DAEMON Fri Jan 16 09:38:45 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhRys-0006wR-RR for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 06:17:38 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhRmG-0004cB-BR for gnunet-developers@gnu.org; Fri, 16 Jan 2004 06:04:36 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhRdR-00033V-1I for gnunet-developers@gnu.org; Fri, 16 Jan 2004 05:56:00 -0500 Received: from [213.51.128.197] (helo=smtpq2.home.nl) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AhRLe-0007sp-Sf for gnunet-developers@gnu.org; Fri, 16 Jan 2004 05:37:07 -0500 Received: from [213.51.128.135] (port=58979 helo=smtp4.home.nl) by smtpq2.home.nl with esmtp (Exim 4.20) id 1AhRLd-0002r0-33; Fri, 16 Jan 2004 11:37:05 +0100 Received: from cc8326-a.ensch1.ov.home.nl ([217.120.174.15]:3866 helo=client) by smtp4.home.nl with smtp (Exim 4.20) id 1AhRLb-0002cW-BE; Fri, 16 Jan 2004 11:37:03 +0100 From: "jan marco alkema" To: "Christian Grothoff" , Date: Fri, 16 Jan 2004 11:37:15 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: <200401081527.13996.grothoff@cs.purdue.edu> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-AtHome-MailScanner-Information: Neem contact op met support@home.nl voor meer informatie X-AtHome-MailScanner: Found to be clean Cc: Subject: [GNUnet-developers] starting up problem? X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: jm.alkema@home.nl List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 11:17:36 -0000 Hello Christian, Maybe a problem in last CSV files? Greetings Jan Marco make[2]: Entering directory `/usr/local/fsh/gnunet/GNUnet' make[2]: Nothing to be done for `install-exec-am'. make[2]: Nothing to be done for `install-data-am'. make[2]: Leaving directory `/usr/local/fsh/gnunet/GNUnet' make[1]: Leaving directory `/usr/local/fsh/gnunet/GNUnet' Jan 16 11:16:43 MESSAGE: gnunetd starting Jan 16 11:16:43 WARNING: scanDirectory /usr/local/fsh/gnunet/var/data/hosts/ returned no known hosts! Jan 16 11:16:43 DEBUG: connection goal is 2048 peers (30000000 BPS bandwidth downstream) Jan 16 11:16:43 Creating new hostkey (this may take a while)... Jan 16 11:17:12 Done creating hostkey. Jan 16 11:17:12 DEBUG: I am 4B3CED38F72FD42D92E9448013ACC5D89077C497 Jan 16 11:17:13 DEBUG: loading applications afs_protocol chat_protocol tbench_protocol tracekit_protocol Jan 16 11:17:13 WARNING: List /usr/local/fsh/gnunet/var/data/afs//database.list of directly shared filenames not available! Jan 16 11:17:13 Could not initialize fileIndex module Jan 16 11:17:13 __BREAK__ at logging.c:240 ./gnunet_install.sh: line 17: 30073 Aborted /usr/local/fsh/gnunet/bin/gnunetd -d -L DEBUG [root@foonsearch gnunet]# Jan 16 11:17:13 DEBUG: too few peers connected, waiting for 1 to reach 16 before trying HTTP download . From MAILER-DAEMON Fri Jan 16 10:04:53 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhVRX-0005nZ-GN for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 09:59:27 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhVRQ-0005mB-PV for gnunet-developers@gnu.org; Fri, 16 Jan 2004 09:59:20 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhUDf-0007kN-Fl for gnunet-developers@gnu.org; Fri, 16 Jan 2004 08:41:35 -0500 Received: from [203.59.3.38] (helo=mail.iinet.net.au) by monty-python.gnu.org with smtp (Exim 4.24) id 1AhTlo-0004Yj-6i for gnunet-developers@gnu.org; Fri, 16 Jan 2004 08:12:16 -0500 Received: (qmail 2741 invoked from network); 16 Jan 2004 12:13:48 -0000 Received: from unknown (HELO localhost) (203.217.36.158) by mail.iinet.net.au with SMTP; 16 Jan 2004 12:13:47 -0000 Date: Fri, 16 Jan 2004 23:14:25 +1100 From: Glenn McGrath To: gnunet-developers@gnu.org Message-Id: <20040116231425.43418a55.bug1@iinet.net.au> X-Mailer: Sylpheed version 0.9.8 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] config file, GNUNETD_HOME and GNUNET_HOME X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 14:59:24 -0000 Hi all, I had a bit of trouble getting gnunet-stats working, i kept getting. # gnunet-stats -c /etc/gnunet.conf Jan 16 22:59:04 Configuration file must specify a directory for GNUnet to store per-peer data under GNUNET_HOME Jan 16 22:59:04 __BREAK__ at logging.c:240 What is the relationship between GNUNET_HOME and GNUNETD_HOME, i see they are both used but only GNUNETD_HOME is defined in the default config file. I defined them both to the same value in the config file and its working now. Glenn From MAILER-DAEMON Fri Jan 16 13:12:46 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhYSc-0005go-8Q for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:12:46 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhYSW-0005f5-BC for gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:12:40 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhYCH-0002o4-Hk for gnunet-developers@gnu.org; Fri, 16 Jan 2004 12:56:25 -0500 Received: from [128.10.2.6] (helo=newman.cs.purdue.edu) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AhXlB-0004sF-P2 for gnunet-developers@gnu.org; Fri, 16 Jan 2004 12:27:53 -0500 Received: from gecko.cs.purdue.edu (gecko.cs.purdue.edu [128.211.1.61]) by newman.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0GHRTVX014952; Fri, 16 Jan 2004 12:27:29 -0500 (EST) Received: from gecko.cs.purdue.edu (localhost [127.0.0.1]) by gecko.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0GHRSdr002512; Fri, 16 Jan 2004 12:27:28 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Christian Grothoff Organization: Secure Software Systems Lab, Computer Science, Purdue University To: Hendrik Pagenhardt , gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] Patch for network load measurement Date: Fri, 16 Jan 2004 12:27:27 -0500 User-Agent: KMail/1.4.3 References: <010462025.20040115152814@gmx.net> <936318510.20040116135914@gmx.net> In-Reply-To: <936318510.20040116135914@gmx.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200401161227.27990.grothoff@cs.purdue.edu> Cc: X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 18:12:42 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday 16 January 2004 07:59 am, Hendrik Pagenhardt wrote: > After testing my code with bandwidth limitations far below my upstream > capacity I found that GNUnet uses to much bandwidth with my new code. > Probably that's why the more pessimistic estimates were implemented. I > spent some time now looking through the parts of the source which > implement the bandwidth allocation strategy I concluded the following: > > - Downstream network load is nowhere used in the code to control > anything (only gnunet-stats). Maybe it could be used to tell peers to > reduce the amount of data sent when its over the limit. Or did I miss a > place were downstream bandwidth control is implemented? Yes, you did. It does not use the load though. Since we can not precisely control what we receive (as opposed to what we send), the code takes the total number of bytes we have available and distributes them among all connected peers (see connection.c). Look at "BufferEntry" with the members recently_received (how much traffic came from peer X), current_connection_value (how much did we appreciate the traffic) and "transmitted_limit" and "idealized limit" (how much is the peer allowed to send us at the moment) and the related code in connection.c. > - Upstream control works be looking at the load and then decide whether > a message of a certain priority can be sent. If the priority is greater > than 2500 it is always sent. I don't know whether is is an unusual high > priority, but it seems suspect. 2500? Where didi you get that from I thought it was ADMIN_PRIORITY (defined to 0xFFFF = 65536). The idea is that certain administrative messages must go through regardless of bandwidth limits (like, say, HANGUP). Now, I also suspect that this priority cap is to blame for the spikes, but I thought that we may be generating too many messages with ADMIN_PRIORITY at times. But if you are right with 2500, that'll be a different story... > - For the upstream control to work effectively it needs a momentous > value estimation of the load. Moving averages are probably not very > suited for the current policy implementation. I'm not sure that they are not. TCP and the OS will smooth things out a bit for us anyway. And we may not have the CPU cycles to always precisely send a message instantly once we're momentously under the limit. Also, given the fact that the messages are of different importance and processing is highly concurrent, I don't see how a momentous value estimation could possibly work as smoothly as the moving averages (and it is important that all changes are smoothed over time to avoid instabilities in the network of the form of quickly going back and forward between low-load (everyone sends) and high-load (nobody sends) states. This type of oscillation can easily happen and is reduced with the moving average but might easily become a problem without one. > I have a few more ideas to make the bandwidth usage more effective and > more predictable, but I will spend some more time trying to understand > the current code before posting more unneeded patches. Btw, I didn't have time to look at the patch yet. C -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQFACB7/9tNtMeXQLkIRAt1wAJ9OAoEJNKfQ4p3QX0PpE5HVQmZ2/wCfTBoF 4ahMyMmIPkhMBDZHnhgtWfM= =+w9g -----END PGP SIGNATURE----- From MAILER-DAEMON Fri Jan 16 13:36:15 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhYpL-0002Be-FI for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:36:15 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhYpF-0002Ah-Pz for gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:36:09 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhYoj-00022X-SL for gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:36:08 -0500 Received: from [212.90.64.119] (helo=smtp1.pp.htv.fi) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AhYfk-0008MV-IC for gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:26:20 -0500 Received: from posti.pp.htv.fi (posti.pp.htv.fi [212.90.64.50]) by smtp1.pp.htv.fi (Postfix) with ESMTP id 7C2FA80209; Fri, 16 Jan 2004 20:26:17 +0200 (EET) Received: from cs71085.pp.htv.fi (cs71085.pp.htv.fi [212.90.71.85]) by posti.pp.htv.fi (8.11.1 (Revision 1.5+JAGae91741+JAGae92668) /8.11.1) with ESMTP id i0GIQHT12800; Fri, 16 Jan 2004 20:26:17 +0200 (EET) Date: Fri, 16 Jan 2004 20:26:41 +0200 (EET) From: Igor Wronsky X-X-Sender: nag@localhost.localdomain To: Hendrik Pagenhardt Subject: Re: [GNUnet-developers] Patch for network load measurement In-Reply-To: <936318510.20040116135914@gmx.net> Message-ID: References: <010462025.20040115152814@gmx.net> <936318510.20040116135914@gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Igor Wronsky List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 18:36:13 -0000 On Fri, 16 Jan 2004, Hendrik Pagenhardt wrote: > - Upstream control works be looking at the load and then decide whether > a message of a certain priority can be sent. If the priority is greater > than 2500 it is always sent. I don't know whether is is an unusual high > priority, but it seems suspect. I have an intuitive feeling that atleast in the case of local requests the node can currently generate unreasonably high priorities (which will very likely also look suspicious to the receivers - this issue might have serious consequences). Christian is likely to have a more accurate view on this. Igor From MAILER-DAEMON Fri Jan 16 14:16:31 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhUjh-0001SD-NN for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 09:14:09 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhUiw-0001OB-CP for gnunet-developers@gnu.org; Fri, 16 Jan 2004 09:13:22 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhTrH-0005gZ-EA for gnunet-developers@gnu.org; Fri, 16 Jan 2004 08:18:26 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AhTfk-0003oo-HN for gnunet-developers@gnu.org; Fri, 16 Jan 2004 08:06:00 -0500 Received: (qmail 32546 invoked by uid 65534); 16 Jan 2004 12:59:14 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp016) with SMTP; 16 Jan 2004 13:59:14 +0100 X-Authenticated: #1791878 Date: Fri, 16 Jan 2004 13:59:14 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <936318510.20040116135914@gmx.net> To: gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] Patch for network load measurement In-Reply-To: <010462025.20040115152814@gmx.net> References: <010462025.20040115152814@gmx.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 14:14:06 -0000 After testing my code with bandwidth limitations far below my upstream capacity I found that GNUnet uses to much bandwidth with my new code. Probably that's why the more pessimistic estimates were implemented. I spent some time now looking through the parts of the source which implement the bandwidth allocation strategy I concluded the following: - Downstream network load is nowhere used in the code to control anything (only gnunet-stats). Maybe it could be used to tell peers to reduce the amount of data sent when its over the limit. Or did I miss a place were downstream bandwidth control is implemented? - Upstream control works be looking at the load and then decide whether a message of a certain priority can be sent. If the priority is greater than 2500 it is always sent. I don't know whether is is an unusual high priority, but it seems suspect. - For the upstream control to work effectively it needs a momentous value estimation of the load. Moving averages are probably not very suited for the current policy implementation. I have a few more ideas to make the bandwidth usage more effective and more predictable, but I will spend some more time trying to understand the current code before posting more unneeded patches. Ciao, Hendrik From MAILER-DAEMON Fri Jan 16 14:20:34 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhZWE-0004eV-I9 for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:20:34 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhZWB-0004do-Hw for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:20:31 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhZVf-0004Xs-0j for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:20:30 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AhZLh-0001vf-NK for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:09:41 -0500 Received: (qmail 24702 invoked by uid 65534); 16 Jan 2004 19:09:40 -0000 Received: from port-ip-213-211-237-69.reverse.mdcc-fun.de (EHLO port-ip-213-211-237-69.reverse.mdcc-fun.de) (213.211.237.69) by mail.gmx.net (mp017) with SMTP; 16 Jan 2004 20:09:40 +0100 X-Authenticated: #1791878 From: Hendrik Pagenhardt To: Christian Grothoff , gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] Patch for network load measurement Date: Fri, 16 Jan 2004 20:09:37 +0100 User-Agent: KMail/1.5.4 References: <010462025.20040115152814@gmx.net> <936318510.20040116135914@gmx.net> <200401161227.27990.grothoff@cs.purdue.edu> In-Reply-To: <200401161227.27990.grothoff@cs.purdue.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401162009.38537.Hendrik.Pagenhardt@gmx.net> Cc: X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik.Pagenhardt@gmx.net List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 19:20:33 -0000 On Friday 16 January 2004 18:27, Christian Grothoff wrote: > Look at "BufferEntry" with the members recently_received (how much > traffic came from peer X), > current_connection_value (how much did we appreciate the traffic) and > "transmitted_limit" and "idealized limit" (how much is the peer allowed > to send us at the moment) and the related code in connection.c. I saw this downstream allocation code shortly after I send off my message. But it still might be useful to take the downstream load into account especially if advanced load measurement is used. It would be quite trivial to reduce the maximum allocateable bandwidth by dividing it by the load if it goes over 100%. > 2500? Where didi you get that from I thought it was ADMIN_PRIORITY > (defined to 0xFFFF = 65536). The idea is that certain administrative > messages must go through regardless of bandwidth limits (like, say, > HANGUP). Now, I also suspect that this priority cap is to blame for > the spikes, but I thought that we may be generating too many messages > with ADMIN_PRIORITY at times. But if you are right with 2500, that'll > be a different story... This comes from server/policy.c "outgoingCheck": ----- if ( (delta < 50) && /* super-fast path */ (priority < 50*50) && /* avoid integer overflow problems with high priorities */ (delta * delta * delta * (priority+1) < 50*50*50) ) { ----- So when the priority is at least 2500 (50*50) the whole condition is false and (independent of load) the message is not dropped. > I'm not sure that they are not. TCP and the OS will smooth things out > a bit for us anyway. And we may not have the CPU cycles to always > precisely send a message instantly once we're momentously under the > limit. Also, given the fact that the messages are of different > importance and processing is highly concurrent, I don't see how a > momentous value estimation could possibly work as smoothly as the > moving averages (and it is important that all changes are smoothed over > time to avoid instabilities in the network of the form of quickly going > back and forward between low-load (everyone sends) and high-load > (nobody sends) states. This type of oscillation can easily happen and > is reduced with the moving average but might easily become a problem > without one. The problem with the moving averages is that the throttling nearly always happens too late. Maybe a "soft" limit would be needed once the (momentanous, based on gnunets internal traffic accounting) load gets above 100% to reduce the frequency with which messages are sent. I'll try some things out and report about my findings. Hendrik From MAILER-DAEMON Fri Jan 16 14:22:02 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhZXd-0004vd-PR for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:22:01 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhZXa-0004vP-UE for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:21:58 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhZNJ-0002Og-59 for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:11:52 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AhZNI-0002O7-9T for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:11:20 -0500 Received: (qmail 22183 invoked by uid 65534); 16 Jan 2004 19:11:19 -0000 Received: from port-ip-213-211-237-69.reverse.mdcc-fun.de (EHLO port-ip-213-211-237-69.reverse.mdcc-fun.de) (213.211.237.69) by mail.gmx.net (mp018) with SMTP; 16 Jan 2004 20:11:19 +0100 X-Authenticated: #1791878 From: Hendrik Pagenhardt To: Igor Wronsky Subject: Re: [GNUnet-developers] Patch for network load measurement Date: Fri, 16 Jan 2004 20:11:17 +0100 User-Agent: KMail/1.5.4 References: <010462025.20040115152814@gmx.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401162011.17530.Hendrik.Pagenhardt@gmx.net> Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik.Pagenhardt@gmx.net List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 19:21:59 -0000 On Friday 16 January 2004 19:18, Igor Wronsky wrote: > However, what I don't understand is the > > if (ret > -10/*lastRet*/) { This is an oversight on my behalf. I wrote that I took out the "slow falloff after peaks" code and this is the place were I did it. I just commented out the old value to compare to (lastRet) and replaced it with -10 to guaranty the direct return of the current value. I just forgot to take the code out completely after debugging, so I'm sorry for the confusion... Hendrik From MAILER-DAEMON Fri Jan 16 14:41:59 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhZhH-0007Vi-Vq for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:31:59 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhZYQ-0005A9-HI for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:22:50 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhZXo-0004xq-7y for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:22:43 -0500 Received: from [128.10.2.6] (helo=newman.cs.purdue.edu) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AhZXn-0004xX-9n for gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:22:11 -0500 Received: from gecko.cs.purdue.edu (gecko.cs.purdue.edu [128.211.1.61]) by newman.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0GJM8VX016573; Fri, 16 Jan 2004 14:22:08 -0500 (EST) Received: from gecko.cs.purdue.edu (localhost [127.0.0.1]) by gecko.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0GJM7dr002944; Fri, 16 Jan 2004 14:22:08 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Christian Grothoff Organization: Secure Software Systems Lab, Computer Science, Purdue University To: Glenn McGrath , gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] config file, GNUNETD_HOME and GNUNET_HOME Date: Fri, 16 Jan 2004 14:22:05 -0500 User-Agent: KMail/1.4.3 References: <20040116231425.43418a55.bug1@iinet.net.au> In-Reply-To: <20040116231425.43418a55.bug1@iinet.net.au> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200401161422.06628.grothoff@cs.purdue.edu> Cc: X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 19:31:57 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 gnunet-stats is a client and thus should take the 'gnunet.user' configuration (~/.gnunet/gnunet.conf by default). GNUNET_HOME is the directory where 'normal' users have their GNUnet files (somewhere in $HOME, by default ~/.gnunet/) - see state.sdb bug reported about GNUnet in the Debian bugtracking system. GNUNETD_HOME is the directory where gnunetd (run as user gnunetd or even root) stores its data (hostkey, AFS-data, host list, etc.; by default /var/gnunet/). If the user running gnunetd is the same as the one running, say, gnunet-gtk, then GNUNETD_HOME==GNUNET_HOME may make sense. The separation was done to allow file-permissions to work nicely. C On Friday 16 January 2004 07:14 am, Glenn McGrath wrote: > Hi all, > > I had a bit of trouble getting gnunet-stats working, i kept getting. > > # gnunet-stats -c /etc/gnunet.conf > Jan 16 22:59:04 Configuration file must specify a directory for GNUnet > to store per-peer data under GNUNET_HOME > Jan 16 22:59:04 __BREAK__ at logging.c:240 > > What is the relationship between GNUNET_HOME and GNUNETD_HOME, i see > they are both used but only GNUNETD_HOME is defined in the default > config file. > > I defined them both to the same value in the config file and its working > now. > > > > Glenn > > > _______________________________________________ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > http://mail.gnu.org/mailman/listinfo/gnunet-developers -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQFACDne9tNtMeXQLkIRAp5xAJoCGZuJ6MZ2rDsWy3l+NhpC4hMyjgCgnM7j 8Yq2tEPMb5HMzMS31uWNUHU= =yejL -----END PGP SIGNATURE----- From MAILER-DAEMON Fri Jan 16 14:59:04 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Aha7U-0005yg-60 for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 14:59:04 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhYxk-0004ja-Tc for gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:44:56 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhYhu-0000dK-Nm for gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:29:05 -0500 Received: from [213.243.153.14] (helo=smtp2.pp.htv.fi) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AhYXd-0006kv-Eh for gnunet-developers@gnu.org; Fri, 16 Jan 2004 13:17:57 -0500 Received: from posti.pp.htv.fi (posti.pp.htv.fi [212.90.64.50]) by smtp2.pp.htv.fi (Postfix) with ESMTP id 081342980C8; Fri, 16 Jan 2004 20:17:55 +0200 (EET) Received: from cs71085.pp.htv.fi (cs71085.pp.htv.fi [212.90.71.85]) by posti.pp.htv.fi (8.11.1 (Revision 1.5+JAGae91741+JAGae92668) /8.11.1) with ESMTP id i0GIHrT07575; Fri, 16 Jan 2004 20:17:54 +0200 (EET) Date: Fri, 16 Jan 2004 20:18:09 +0200 (EET) From: Igor Wronsky X-X-Sender: nag@localhost.localdomain To: Hendrik Pagenhardt Subject: Re: [GNUnet-developers] Patch for network load measurement In-Reply-To: <010462025.20040115152814@gmx.net> Message-ID: References: <010462025.20040115152814@gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: gnunet-developers@gnu.org X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Igor Wronsky List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 19:58:57 -0000 On Thu, 15 Jan 2004, Hendrik Pagenhardt wrote: > This time a context diff is provided to help evaluating the changes by > the core developers. I've looked at it (despite the latest message you sent). To be sure your "ret" calculations were correct, I would probably have had to take up pen and paper, and I didn't bother to do that. Maybe there is no need to be so concise in the future. Anyway, at a glance they seem otherwise ok'ish. The few new variables added (.*TrafficLastProc) would probably still benefit from comments, including units used and the idea in their calculation. However, what I don't understand is the if (ret > -10/*lastRet*/) { part. In addition to the comment there being in obscure place (I'd prefer separate like /* line this and that */), I don't understand what is the significance of -10? I gather that the routines should return -1 on failure and a positive number rest of the time. This reminds me that the previous routines are probably handling the error situation incorrectly as well. Igor From MAILER-DAEMON Fri Jan 16 17:51:11 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AhcnS-0003B1-7s for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 17:50:34 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhcnH-0002pc-Al for gnunet-developers@gnu.org; Fri, 16 Jan 2004 17:50:23 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Ahcmb-0001pI-DF for gnunet-developers@gnu.org; Fri, 16 Jan 2004 17:50:15 -0500 Received: from [128.10.2.6] (helo=newman.cs.purdue.edu) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AhcmN-0001c1-Lq for gnunet-developers@gnu.org; Fri, 16 Jan 2004 17:49:27 -0500 Received: from gecko.cs.purdue.edu (gecko.cs.purdue.edu [128.211.1.61]) by newman.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0GMnQVX020221 for ; Fri, 16 Jan 2004 17:49:27 -0500 (EST) Received: from gecko.cs.purdue.edu (localhost [127.0.0.1]) by gecko.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0GMnQdr010104 for ; Fri, 16 Jan 2004 17:49:26 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Christian Grothoff Organization: Secure Software Systems Lab, Computer Science, Purdue University To: gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] starting up problem? Date: Fri, 16 Jan 2004 17:49:25 -0500 User-Agent: KMail/1.4.3 References: In-Reply-To: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200401161749.25782.grothoff@cs.purdue.edu> X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 22:50:32 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Right, fopen(...,"a+") fails if the directory does not exist -- and for a new setup some recent unrelated shuffling of the initialization sequence made it possible for the directory to not yet be there. Fixed, thanks for reporting! Christian On Friday 16 January 2004 05:37 am, jan marco alkema wrote: > Hello Christian, > > Maybe a problem in last CSV files? > > Greetings Jan Marco > > > make[2]: Entering directory `/usr/local/fsh/gnunet/GNUnet' > make[2]: Nothing to be done for `install-exec-am'. > make[2]: Nothing to be done for `install-data-am'. > make[2]: Leaving directory `/usr/local/fsh/gnunet/GNUnet' > make[1]: Leaving directory `/usr/local/fsh/gnunet/GNUnet' > Jan 16 11:16:43 MESSAGE: gnunetd starting > Jan 16 11:16:43 WARNING: scanDirectory > /usr/local/fsh/gnunet/var/data/hosts/ returned no known hosts! > Jan 16 11:16:43 DEBUG: connection goal is 2048 peers (30000000 BPS > bandwidth downstream) > Jan 16 11:16:43 Creating new hostkey (this may take a while)... > Jan 16 11:17:12 Done creating hostkey. > Jan 16 11:17:12 DEBUG: I am 4B3CED38F72FD42D92E9448013ACC5D89077C497 > Jan 16 11:17:13 DEBUG: loading applications afs_protocol chat_protocol > tbench_protocol tracekit_protocol > Jan 16 11:17:13 WARNING: List > /usr/local/fsh/gnunet/var/data/afs//database.list of directly shared > filenames not available! > Jan 16 11:17:13 Could not initialize fileIndex module > Jan 16 11:17:13 __BREAK__ at logging.c:240 > ./gnunet_install.sh: line 17: 30073 Aborted > /usr/local/fsh/gnunet/bin/gnunetd -d -L DEBUG > [root@foonsearch gnunet]# Jan 16 11:17:13 DEBUG: too few peers connected, > waiting for 1 to reach 16 before trying HTTP download . > > > > > _______________________________________________ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > http://mail.gnu.org/mailman/listinfo/gnunet-developers -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQFACGp19tNtMeXQLkIRAtjYAJ43m/PjAFP5ShlSwldWhFFPkMcjZwCeJXVs KEAl5/jpNB7f+Ib+W8uRgQU= =GdEz -----END PGP SIGNATURE----- From MAILER-DAEMON Fri Jan 16 18:53:38 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Ahdlh-0002Fw-QM for mharc-gnunet-developers@gnu.org; Fri, 16 Jan 2004 18:52:49 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AhdlM-0001YB-M8 for gnunet-developers@gnu.org; Fri, 16 Jan 2004 18:52:28 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AhdkZ-0000qI-8D for gnunet-developers@gnu.org; Fri, 16 Jan 2004 18:52:10 -0500 Received: from [203.59.3.37] (helo=mail.iinet.net.au) by monty-python.gnu.org with smtp (Exim 4.24) id 1AhdkX-0000o1-Md for gnunet-developers@gnu.org; Fri, 16 Jan 2004 18:51:37 -0500 Received: (qmail 15817 invoked from network); 16 Jan 2004 23:49:01 -0000 Received: from unknown (HELO localhost) (203.217.33.115) by mail.iinet.net.au with SMTP; 16 Jan 2004 23:49:00 -0000 Date: Sat, 17 Jan 2004 10:49:37 +1100 From: Glenn McGrath To: gnunet-developers@gnu.org Message-Id: <20040117104937.22dcf038.bug1@iinet.net.au> In-Reply-To: <20040116231425.43418a55.bug1@iinet.net.au> References: <20040116231425.43418a55.bug1@iinet.net.au> X-Mailer: Sylpheed version 0.9.8 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] multiple users per host, GNUNETD_HOME and GNUNET_HOME X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 23:52:46 -0000 On Fri, 16 Jan 2004 23:14:25 +1100 Glenn McGrath wrote: > Hi all, > > I had a bit of trouble getting gnunet-stats working, i kept getting. > > # gnunet-stats -c /etc/gnunet.conf > Jan 16 22:59:04 Configuration file must specify a directory for GNUnet > to store per-peer data under GNUNET_HOME > Jan 16 22:59:04 __BREAK__ at logging.c:240 > > What is the relationship between GNUNET_HOME and GNUNETD_HOME, i see > they are both used but only GNUNETD_HOME is defined in the default > config file. So, GNUNETD_HOME is set in the global config file, and GNUNET_HOME is set in the per user config file. As regular user i setup a ~/.gnunet/gnunet.conf that specified GNUNET_HOME and gnunet-stats is working now. I have gnunetd started as root, the resulting hostkey in $GNUNETD_HOME/.hostkey has permissions 600 which means it isnt readable to normal users. I now have problems with gnunet-peer-info. $ gnunet-peer-info Jan 17 10:17:53 Creating new hostkey (this may take a while)... Jan 17 10:18:28 WARNING: Writing 914 bytes to file /var/lib/GNUnet/.hostkey failed! Jan 17 10:18:28 INFO: error closing file descriptor in storage.c:439 (Bad file descriptor) (then a list of peers follow) If i made a gnunet group and make .hostkey's permission 0640, then gnunet-peer-info would work better, does this sound like a good idea ? Or perhaps gnunet-peer-info should check permissions of the file before trying to replace it. The hostkey is supposed to private as it can compromise anonymity, so i want to be carefull about this, also its probably better to setup gnunet to run as a non-root user/group. Glenn From MAILER-DAEMON Sat Jan 17 18:38:30 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Ai01N-0004Tq-Sq for mharc-gnunet-developers@gnu.org; Sat, 17 Jan 2004 18:38:29 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Ai01L-0004TG-VN for gnunet-developers@gnu.org; Sat, 17 Jan 2004 18:38:27 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Ai00p-0004Ns-9e for gnunet-developers@gnu.org; Sat, 17 Jan 2004 18:38:27 -0500 Received: from [203.59.3.35] (helo=mail.iinet.net.au) by monty-python.gnu.org with smtp (Exim 4.24) id 1Ai00o-0004ND-Rp for gnunet-developers@gnu.org; Sat, 17 Jan 2004 18:37:55 -0500 Received: (qmail 18950 invoked from network); 17 Jan 2004 23:35:27 -0000 Received: from unknown (HELO localhost) (203.217.33.115) by mail.iinet.net.au with SMTP; 17 Jan 2004 23:35:23 -0000 Date: Sun, 18 Jan 2004 10:35:53 +1100 From: Glenn McGrath To: gnunet-developers@gnu.org Message-Id: <20040118103553.45764970.bug1@iinet.net.au> X-Mailer: Sylpheed version 0.9.8 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] debian packages of 0.6.1a available X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2004 23:38:28 -0000 Hello all, Ive finally got some 0.6.1a debian packages available, they are built against libgcrypt at last, they should be in your local debian mirror now. Let me know if you have any problems with them. Thanks Glenn From MAILER-DAEMON Sun Jan 18 16:03:30 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AiK4w-00031Y-24 for mharc-gnunet-developers@gnu.org; Sun, 18 Jan 2004 16:03:30 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AiK4u-000304-0D for gnunet-developers@gnu.org; Sun, 18 Jan 2004 16:03:28 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AiK1X-00010i-MI for gnunet-developers@gnu.org; Sun, 18 Jan 2004 16:00:30 -0500 Received: from [204.127.203.184] (helo=sccmmhc02.asp.att.net) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AiK02-00005K-TG for gnunet-developers@gnu.org; Sun, 18 Jan 2004 15:58:26 -0500 Received: from elma (12-222-155-71.client.insightbb.com[12.222.155.71]) by sccmmhc02.asp.att.net (sccmmhc02) with SMTP id <20040118205825mm2009qqvme>; Sun, 18 Jan 2004 20:58:25 +0000 From: Christian Grothoff Organization: Purdue University, Department of Computer Science To: gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] multiple users per host, GNUNETD_HOME and GNUNET_HOME Date: Sun, 18 Jan 2004 15:59:02 -0500 User-Agent: KMail/1.5.4 References: <20040116231425.43418a55.bug1@iinet.net.au> <20040117104937.22dcf038.bug1@iinet.net.au> In-Reply-To: <20040117104937.22dcf038.bug1@iinet.net.au> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401181559.02009.grothoff@cs.purdue.edu> X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Jan 2004 21:03:28 -0000 > > What is the relationship between GNUNET_HOME and GNUNETD_HOME, i see > > they are both used but only GNUNETD_HOME is defined in the default > > config file. > > So, GNUNETD_HOME is set in the global config file, and GNUNET_HOME is > set in the per user config file. Right. > As regular user i setup a ~/.gnunet/gnunet.conf that specified > GNUNET_HOME and gnunet-stats is working now. > > I have gnunetd started as root, the resulting hostkey in > $GNUNETD_HOME/.hostkey has permissions 600 which means it isnt readable > to normal users. > > I now have problems with gnunet-peer-info. Normal users should have no reason to run gnunet-peer-info, it is just a diagnostics tool for gnunetd. > $ gnunet-peer-info > Jan 17 10:17:53 Creating new hostkey (this may take a while)... > Jan 17 10:18:28 WARNING: Writing 914 bytes to file > /var/lib/GNUnet/.hostkey failed! > Jan 17 10:18:28 INFO: error closing file descriptor in storage.c:439 > (Bad file descriptor) > > (then a list of peers follow) > > > If i made a gnunet group and make .hostkey's permission 0640, then > gnunet-peer-info would work better, does this sound like a good idea ? You'd also have to SGID gnunet-peer-info. I'm not sure that's such a great idea. > Or perhaps gnunet-peer-info should check permissions of the file before > trying to replace it. That is true, only that this specific piece of code at this point currently does not know that it is not run by gnunetd. But that should be fixed. > The hostkey is supposed to private as it can compromise anonymity, so i > want to be carefull about this, also its probably better to setup gnunet > to run as a non-root user/group. Right. C From MAILER-DAEMON Thu Jan 22 03:40:00 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AjaNR-0008Lj-M3 for mharc-gnunet-developers@gnu.org; Thu, 22 Jan 2004 03:39:49 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AjaNO-0008I9-Ra for gnunet-developers@gnu.org; Thu, 22 Jan 2004 03:39:46 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AjaMn-0007e2-R4 for gnunet-developers@gnu.org; Thu, 22 Jan 2004 03:39:42 -0500 Received: from [213.165.64.20] (helo=mail.gmx.net) by monty-python.gnu.org with smtp (Exim 4.24) id 1AjaMl-0007Lp-UB for gnunet-developers@gnu.org; Thu, 22 Jan 2004 03:39:08 -0500 Received: (qmail 18321 invoked by uid 65534); 22 Jan 2004 08:39:00 -0000 Received: from port-213-148-152-104.reverse.qsc.de (EHLO zaphod.chromasphere.com) (213.148.152.104) by mail.gmx.net (mp027) with SMTP; 22 Jan 2004 09:39:00 +0100 X-Authenticated: #1791878 Date: Thu, 22 Jan 2004 09:38:59 +0100 From: Hendrik Pagenhardt X-Mailer: The Bat! (v1.62r) Personal X-Priority: 3 (Normal) Message-ID: <4685371.20040122093859@gmx.net> To: gnunet-developers@gnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [GNUnet-developers] Query TTL question X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: Hendrik Pagenhardt List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2004 08:39:47 -0000 Hello, I noticed quite high numbers in the statistics of my gnunetd regarding p2p namespace queries: Uptime (seconds) : 76770 # p2p queries sent : 2312962 # p2p queries received : 500532 # p2p super queries received : 330263 # p2p CHK content received (kb) : 162462 # p2p search results received (kb) : 90859 # client queries received : 13443 # client CHK content inserted (kb) : 0 # client 3HASH search results inserted (kb) : 0 # client file index requests received : 0 # file index requests received : 0 # super query index requests received : 0 # client CHK content deleted (kb) : 0 # client 3HASH search results deleted (kb) : 0 # client file unindex requests received : 0 # file unindex requests received : 0 # super query unindex requests received : 0 # client SBlock insert requests received : 0 # client namespace queries received : 0 # p2p namespace queries received : 14142350 # p2p SBlocks received : 81 So that means my node receives about 184 namespace queries per second. Compared to the other query numbers this appears very high. Is this normal behavior? And if yes, why are so many NS queries floating the network? I looked through the sources then and found the following in handler.c: if (ttl < 0) { ttl = ttl - 2*TTL_DECREMENT - randomi(TTL_DECREMENT); if (ttl > 0) return OK; /* just abort */ } else ttl = ttl - 2*TTL_DECREMENT - randomi(TTL_DECREMENT); This is part of "handleNSQUERY" and "handleQUERY" and is executed before the query is evaluated. When I read it correctly, it decrements the TTL by between twice and three times TTL_DECREMENT. When the TTL was negative before, it is tested after the decrement to be positive, and if true handling the query is aborted. In effect this means that when an underflow of the TTL occurs, it is dropped? And only then? Shouldn't it be dropped when the TTL becomes negative with a certain probability? And there is a difference between "handleNSQUERY" and "handleQUERY": /* adjust TTL */ if ( (ttl > 0) && (ttl > (int)(prio+3)*TTL_DECREMENT) ) ttl = (int) (prio+3)*TTL_DECREMENT; /* bound! */ This is only used in "handleQUERY", but I think it should be done also for namespace queries... Ciao, Hendrik From MAILER-DAEMON Thu Jan 22 14:54:51 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Ajkuh-00038F-Nz for mharc-gnunet-developers@gnu.org; Thu, 22 Jan 2004 14:54:51 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Ajkue-00037B-OQ for gnunet-developers@gnu.org; Thu, 22 Jan 2004 14:54:48 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AjknJ-0000tQ-Nr for gnunet-developers@gnu.org; Thu, 22 Jan 2004 14:47:46 -0500 Received: from [128.10.2.6] (helo=newman.cs.purdue.edu) by monty-python.gnu.org with esmtp (Exim 4.24) id 1Ajkm2-0008Rm-Ea for gnunet-developers@gnu.org; Thu, 22 Jan 2004 14:45:54 -0500 Received: from gecko.cs.purdue.edu (gecko.cs.purdue.edu [128.211.1.61]) by newman.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0MJjlVX022487 for ; Thu, 22 Jan 2004 14:45:48 -0500 (EST) Received: from gecko.cs.purdue.edu (localhost [127.0.0.1]) by gecko.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0MJjldr024163 for ; Thu, 22 Jan 2004 14:45:47 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Christian Grothoff Organization: Secure Software Systems Lab, Computer Science, Purdue University To: gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] Query TTL question Date: Thu, 22 Jan 2004 14:45:46 -0500 User-Agent: KMail/1.4.3 References: <4685371.20040122093859@gmx.net> In-Reply-To: <4685371.20040122093859@gmx.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200401221445.46904.grothoff@cs.purdue.edu> X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2004 19:54:49 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 22 January 2004 03:38 am, Hendrik Pagenhardt wrote: > Hello, > > I noticed quite high numbers in the statistics of my gnunetd regarding > p2p namespace queries: > > > > So that means my node receives about 184 namespace queries per second. > Compared to the other query numbers this appears very high. Is this > normal behavior? And if yes, why are so many NS queries floating the > network? There's a bug, Igor has already (in private mail) reported the same behavior. > I looked through the sources then and found the following in handler.c: > > if (ttl < 0) { > ttl = ttl - 2*TTL_DECREMENT - randomi(TTL_DECREMENT); > if (ttl > 0) > return OK; /* just abort */ > } else > ttl = ttl - 2*TTL_DECREMENT - randomi(TTL_DECREMENT); > > This is part of "handleNSQUERY" and "handleQUERY" and is executed before > the query is evaluated. When I read it correctly, it decrements the TTL > by between twice and three times TTL_DECREMENT. When the TTL was > negative before, it is tested after the decrement to be positive, and if > true handling the query is aborted. In effect this means that when an > underflow of the TTL occurs, it is dropped? And only then? Shouldn't it > be dropped when the TTL becomes negative with a certain probability? No, drops are relative to other TTLs in the system and happen in routing.c, they don't probabilistically depend on the absolute TTL value but only on the TTL relative to other TTLs. The underflow handling is just there to avoid an attack where I send MIN_INT, you check that it is 'not unreasonably large', then decrement it (get a very large TTL due to underflow) and then route that query 'indefinitely'. The underflow should never happen in 'normal' operation. > And there is a difference between "handleNSQUERY" and "handleQUERY": > > /* adjust TTL */ > if ( (ttl > 0) && > (ttl > (int)(prio+3)*TTL_DECREMENT) ) > ttl = (int) (prio+3)*TTL_DECREMENT; /* bound! */ > > This is only used in "handleQUERY", but I think it should be done also > for namespace queries... Right, the handling of the two should always be symmetric, and while this one should not result in the 100 NS-query-per-second problem (Igor's investigation so far seems to point at routing.c) it should be fixed also. Christian -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQFAEChq9tNtMeXQLkIRAuMcAJ9jG0eKNaZ7xk+m3angLFz5V6PqZQCdFdDC QjKh//mzAV/XoL8Aw6F4Xrs= =hMEx -----END PGP SIGNATURE----- From MAILER-DAEMON Fri Jan 23 11:21:09 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Ak43R-0003iH-Jf for mharc-gnunet-developers@gnu.org; Fri, 23 Jan 2004 11:21:09 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Ajhp6-0003L5-BK for gnunet-developers@gnu.org; Thu, 22 Jan 2004 11:36:52 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AjhoU-00030J-04 for gnunet-developers@gnu.org; Thu, 22 Jan 2004 11:36:46 -0500 Received: from [128.214.9.1] (helo=mail.cs.helsinki.fi) by monty-python.gnu.org with esmtp (TLSv1:DES-CBC3-SHA:168) (Exim 4.24) id 1AjhoP-0002D8-EU for gnunet-developers@gnu.org; Thu, 22 Jan 2004 11:36:09 -0500 Received: from bogrund-15.cs.helsinki.fi (bogrund-15.cs.helsinki.fi [::ffff:128.214.48.15]) (IDENT: root, AUTH: PLAIN cs-relay, TLS: TLSv1/SSLv3, 168bits, DES-CBC3-SHA) by mail.cs.helsinki.fi with esmtp; Thu, 22 Jan 2004 18:35:36 +0200 Received: from bogrund-15.cs.helsinki.fi (localhost [127.0.0.1]) by bogrund-15.cs.helsinki.fi (8.12.8/8.12.8) with ESMTP id i0MGZarJ010324 for ; Thu, 22 Jan 2004 18:35:36 +0200 Received: from localhost (jtlindgr@localhost) by bogrund-15.cs.helsinki.fi (8.12.8/8.12.8/Submit) with ESMTP id i0MGZZXP010320 for ; Thu, 22 Jan 2004 18:35:36 +0200 X-Authentication-Warning: bogrund-15.cs.helsinki.fi: jtlindgr owned process doing -bs Date: Thu, 22 Jan 2004 18:35:35 +0200 (EET) From: "J. Lindgren" To: gnunet-developers@gnu.org Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 23 Jan 2004 11:21:08 -0500 Subject: [GNUnet-developers] GNUnet-DHT project X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2004 16:37:19 -0000 Greetings, everyone, we have started a DHT-on-GNUNET software engineering course project at the Computer Science Dept., University of Helsinki. Besides playing a project, the aim is to produce a Distributed Hash Table implementation for the GNUnet platform. The implementation will probably be something on the lines of Kademlia/Pastry/Chord and atleast offer a general API that other GNUnet applications can use for their purposes (such as host lookup). Our intention is that the resulting GPL'd code will be linked to the GNUnet source tree when completed (early summer, probably) after which it will enter the typical GNUnet free-4-all development. Before that, it will most likely be developed on our departments CVS. However, the group might have general DHT/P2P-related questions that could require a deeper understanding of the various concepts and their implications than I personally have. Due to this, we have agreed with Christian that its ok to use the gnunet-developers@ list for DHT related questions/discussions etc should people like to do that. There's also a DHT mantis category now, but it will probably be useful only to the group at this point. So considering that there is probably no single correct answer for anything, if anyone has any opinions or useful knowledge, don't hesitate to give input. Good hints/suggestions will be especially useful in the early phase of the project, as the group will very likely be forced to something like a traditional waterfall development model. This roughly means that once something is specified, they are likely to keep it that way. Nevertheless, some good might as well come out of it. ;D Cheers, J. From MAILER-DAEMON Sun Jan 25 08:32:14 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AkkN4-0004SW-9f for mharc-gnunet-developers@gnu.org; Sun, 25 Jan 2004 08:32:14 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AkkN1-0004SG-Gv for gnunet-developers@gnu.org; Sun, 25 Jan 2004 08:32:11 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AkkMV-0004LL-Ro for gnunet-developers@gnu.org; Sun, 25 Jan 2004 08:32:10 -0500 Received: from [213.51.128.196] (helo=smtpq1.home.nl) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AkkMV-0004LH-7n for gnunet-developers@gnu.org; Sun, 25 Jan 2004 08:31:39 -0500 Received: from [213.51.128.132] (port=44826 helo=smtp1.home.nl) by smtpq1.home.nl with esmtp (Exim 4.20) id 1AkkMT-0000jM-Mn for gnunet-developers@gnu.org; Sun, 25 Jan 2004 14:31:37 +0100 Received: from cc8326-a.ensch1.ov.home.nl ([217.120.174.15]:2880 helo=client) by smtp1.home.nl with smtp (Exim 4.20) id 1AkkMS-00000a-Kw for gnunet-developers@gnu.org; Sun, 25 Jan 2004 14:31:36 +0100 From: "jan marco alkema" To: Date: Sun, 25 Jan 2004 14:32:12 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-AtHome-MailScanner-Information: Neem contact op met support@home.nl voor meer informatie X-AtHome-MailScanner: Found to be clean Subject: [GNUnet-developers] Visual Studio "port programming" X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list Reply-To: jm.alkema@home.nl List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jan 2004 13:32:12 -0000 Hello Christian, I compiled wget with Visual studio. Appendix A contains the Visual C port programming procedures. All these procedures are in ws2_32.lib In Visual studio you should comment the path "C:\Program Files\Microsoft Visual Studio .NET\Vc7\PlatformSDK\lib\Ws2_32.lib" else you get the error "LINK : fatal error LNK1181: cannot open input file 'C:\Program.obj'". The next link contains useful specific Linux -> Visual C port info: http://mywebpage.netscape.com/yongweiwu/time.htm A possible example can be found at: http://www.nexit.no/wsp.html I will studie this example for the port of gnunetd to Visual Studio next week, Greetings Jan Marco Appendix A: fshwget.obj : error LNK2019: unresolved external symbol __imp__WSAStartup@8 referenced in function _ws_startup Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcewinsk/html/_wcesdk_wsastartup.asp Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/wsastartup_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__WSACleanup@0 referenced in function _ws_cleanup Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/wsacleanup_2.asp Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcewinsk/html/_wcesdk_wsacleanup.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__WSAGetLastError@0 referenced in function _thread_helper@4 Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/wsagetlasterror_2.as p Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcewinsk/html/_wcesdk_wsagetlasterro r.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__htons@4 referenced in function _wget_sockaddr_set_address Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcewinsk/html/_wcesdk_htons.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__gethostbyname@4 referenced in function _gethostbyname_with_timeout_callback Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcewinsk/html/_wcesdk_gethostbyname. asp fshwget.obj : error LNK2019: unresolved external symbol __imp__inet_ntoa@4 referenced in function _pretty_print_address Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcewinsk/html/_wcesdk_inet_ntoa.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__closesocket@4 referenced in function _closeport Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/closesocket_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__getsockname@12 referenced in function _conaddr Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcebluet/html/cecongetsockname.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__select@20 referenced in function _test_socket_open Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/select_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__connect@12 referenced in function _connect_with_timeout_callback Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/connect_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__inet_addr@4 referenced in function _lookup_host Header: Winsock2.h. Link Library: Ws2.lib. http://msdn.microsoft.com/library/en-us/wcewinsk/html/_wcesdk_inet_addr.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__recv@16 referenced in function _iread Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/recv_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__send@16 referenced in function _iwrite Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/send_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__accept@12 referenced in function _acceptport Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/accept_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__listen@8 referenced in function _bindport Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/listen_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__bind@12 referenced in function _bindport Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/bind_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__setsockopt@20 referenced in function _bindport Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/setsockopt_2.asp fshwget.obj : error LNK2019: unresolved external symbol __imp__socket@12 referenced in function _bindport Header: Declared in Winsock2.h. Library: Use Ws2_32.lib. http://msdn.microsoft.com/library/en-us/winsock/winsock/socket_2.asp From MAILER-DAEMON Mon Jan 26 04:29:31 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Al33j-0006ME-3Y for mharc-gnunet-developers@gnu.org; Mon, 26 Jan 2004 04:29:31 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Al33g-0006Kz-5u for gnunet-developers@gnu.org; Mon, 26 Jan 2004 04:29:28 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Al338-00069u-6G for gnunet-developers@gnu.org; Mon, 26 Jan 2004 04:29:26 -0500 Received: from [131.174.191.234] (helo=charogne.sci.kun.nl) by monty-python.gnu.org with esmtp (Exim 4.24) id 1Al31s-0005Hc-Ec for gnunet-developers@gnu.org; Mon, 26 Jan 2004 04:27:36 -0500 Received: from loic by charogne.sci.kun.nl with local (Exim 3.35 #1 (Debian)) id 1Al31S-0007P4-00 for ; Mon, 26 Jan 2004 10:27:10 +0100 Date: Mon, 26 Jan 2004 10:27:10 +0100 From: =?iso-8859-15?Q?Lo=EFc?= Le Guyader To: gnunet-developers@gnu.org Message-ID: <20040126092710.GA28360@charogne.sci.kun.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HcAYCG3uE/tztfnV" Content-Disposition: inline User-Agent: Mutt/1.3.28i X-Face: .3?y02Mk0JM(iZFU*$ncpy=:kM>3T^eKs, x}Z_B+@%dK)NtDk=ff[yHxrm%x6'Nyv>A&*8r $]0A@RL*7j}"M81{)(p7%^|0*dLyl`6jd).&0c?Rgkh@W%b8d%dd^r}p_UX*IA4+7$$d=#5t.r:4>w nH, sNTo`c_ev5lb, rHC*uoWD=5>mtH\#!|tJu9.&tvOmF7CxWqOvK%fWpdow0&NlFzAT6L Sender: Loic Le Guyader Subject: [GNUnet-developers] FREE NULL in keywords.c X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2004 09:29:29 -0000 --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi all, I got this, wathever I try to insert: loic@charogne ~$ gnunet-insert -k pouet GPL=20 Jan 26 10:23:10 FREE called with NULL ptr at keywords.c:26 Jan 26 10:23:10 __BREAK__ at logging.c:240 Abandon loic@charogne ~$ gnunet-insert GPL=20 Jan 26 10:25:28 FREE called with NULL ptr at keywords.c:26 Jan 26 10:25:28 __BREAK__ at logging.c:240 Abandon It's the 0.6.1a recompiled (backported) on Debian stable. --=20 JabberID: loico@jabber.org Visit the "Toy Story" Web Page! =20 --HcAYCG3uE/tztfnV Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: Pour information voir http://www.gnupg.org iD8DBQFAFN1ucKVi1KED/0IRAosrAJ95AxcvmKggXRJQZIje7B3b/WPJ1QCdGX+A /yWflxbEWYHjiKayYwgOmiA= =WhPr -----END PGP SIGNATURE----- --HcAYCG3uE/tztfnV-- From MAILER-DAEMON Mon Jan 26 09:35:55 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Al7qF-0001rV-4d for mharc-gnunet-developers@gnu.org; Mon, 26 Jan 2004 09:35:55 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Al7qC-0001px-2T for gnunet-developers@gnu.org; Mon, 26 Jan 2004 09:35:52 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1Al7pf-0001e4-KB for gnunet-developers@gnu.org; Mon, 26 Jan 2004 09:35:50 -0500 Received: from [131.174.191.234] (helo=charogne.sci.kun.nl) by monty-python.gnu.org with esmtp (Exim 4.24) id 1Al5nL-0003fn-Sd for gnunet-developers@gnu.org; Mon, 26 Jan 2004 07:24:48 -0500 Received: from loic by charogne.sci.kun.nl with local (Exim 3.35 #1 (Debian)) id 1Al5nD-0004P9-00 for ; Mon, 26 Jan 2004 13:24:39 +0100 Date: Mon, 26 Jan 2004 13:24:39 +0100 From: =?iso-8859-15?Q?Lo=EFc?= Le Guyader To: gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] FREE NULL in keywords.c Message-ID: <20040126122437.GA16870@charogne.sci.kun.nl> References: <20040126092710.GA28360@charogne.sci.kun.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IS0zKkzwUGydFO0o" Content-Disposition: inline In-Reply-To: <20040126092710.GA28360@charogne.sci.kun.nl> User-Agent: Mutt/1.3.28i X-Face: .3?y02Mk0JM(iZFU*$ncpy=:kM>3T^eKs, x}Z_B+@%dK)NtDk=ff[yHxrm%x6'Nyv>A&*8r $]0A@RL*7j}"M81{)(p7%^|0*dLyl`6jd).&0c?Rgkh@W%b8d%dd^r}p_UX*IA4+7$$d=#5t.r:4>w nH, sNTo`c_ev5lb, rHC*uoWD=5>mtH\#!|tJu9.&tvOmF7CxWqOvK%fWpdow0&NlFzAT6L Sender: Loic Le Guyader X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2004 14:35:53 -0000 --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Le 26 janvier 2004, Lo=EFc Le Guyader, =E0 bout, prit son clavier pour taper sur son =E9cran: > Hi all, > I got this, wathever I try to insert: > loic@charogne ~$ gnunet-insert -k pouet GPL=20 > Jan 26 10:23:10 FREE called with NULL ptr at keywords.c:26 > Jan 26 10:23:10 __BREAK__ at logging.c:240 > Abandon Ok, the problem was that I hadn't specify EXTRACTORS =3D libextractor_filename:-libextractor_split:-libextractor_lower in my personnal gnunet.conf file. Sorry. --=20 JabberID: loico@jabber.org Toy Story: > US$174M domestic box office receipts and an Oscar so far. --IS0zKkzwUGydFO0o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: Pour information voir http://www.gnupg.org iD8DBQFAFQcEcKVi1KED/0IRAg+qAJ9ia9/mBny7Vn+JKPdCjJLzqJ+UYQCghc3K mSC6JoSJLggC2KVbr1Vse94= =HYMD -----END PGP SIGNATURE----- --IS0zKkzwUGydFO0o-- From MAILER-DAEMON Mon Jan 26 17:20:26 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AlF5l-0003ZS-Pg for mharc-gnunet-developers@gnu.org; Mon, 26 Jan 2004 17:20:25 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AlF5c-0003Xr-RT for gnunet-developers@gnu.org; Mon, 26 Jan 2004 17:20:16 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AlF55-0003Ky-EK for gnunet-developers@gnu.org; Mon, 26 Jan 2004 17:20:14 -0500 Received: from [128.10.2.6] (helo=newman.cs.purdue.edu) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AlF52-0003Ia-HC for gnunet-developers@gnu.org; Mon, 26 Jan 2004 17:19:40 -0500 Received: from gecko.cs.purdue.edu (gecko.cs.purdue.edu [128.211.1.61]) by newman.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0QMJYVX026398; Mon, 26 Jan 2004 17:19:34 -0500 (EST) Received: from gecko.cs.purdue.edu (localhost [127.0.0.1]) by gecko.cs.purdue.edu (8.12.9/8.12.9/PURDUE_CS-2.0) with ESMTP id i0QMJWdr013022; Mon, 26 Jan 2004 17:19:33 -0500 Content-Type: text/plain; charset="iso-8859-15" From: Christian Grothoff Organization: Secure Software Systems Lab, Computer Science, Purdue University To: =?iso-8859-15?q?Lo=EFc=20Le=20Guyader?= , gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] FREE NULL in keywords.c Date: Mon, 26 Jan 2004 17:19:32 -0500 User-Agent: KMail/1.4.3 References: <20040126092710.GA28360@charogne.sci.kun.nl> In-Reply-To: <20040126092710.GA28360@charogne.sci.kun.nl> MIME-Version: 1.0 Message-Id: <200401261719.32494.grothoff@cs.purdue.edu> Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by newman.cs.purdue.edu id i0QMJYVX026398 Cc: X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2004 22:20:18 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Fixed in CVS. On Monday 26 January 2004 04:27 am, Lo=EFc Le Guyader wrote: > Hi all, > I got this, wathever I try to insert: > loic@charogne ~$ gnunet-insert -k pouet GPL > Jan 26 10:23:10 FREE called with NULL ptr at keywords.c:26 > Jan 26 10:23:10 __BREAK__ at logging.c:240 > Abandon > loic@charogne ~$ gnunet-insert GPL > Jan 26 10:25:28 FREE called with NULL ptr at keywords.c:26 > Jan 26 10:25:28 __BREAK__ at logging.c:240 > Abandon > > It's the 0.6.1a recompiled (backported) on Debian stable. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQFAFZJ09tNtMeXQLkIRArcZAJ92sC3O6yFh+7F12PTKO1z587E34ACfSD1+ 1mo/iRSlbeK9V0Tl7AJdD50=3D =3DpLix -----END PGP SIGNATURE----- From MAILER-DAEMON Sat Jan 31 04:54:42 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1Amrpq-0005rm-B7 for mharc-gnunet-developers@gnu.org; Sat, 31 Jan 2004 04:54:42 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1Amrpo-0005r7-3q for gnunet-developers@gnu.org; Sat, 31 Jan 2004 04:54:40 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AmrpS-0005jb-VJ for gnunet-developers@gnu.org; Sat, 31 Jan 2004 04:54:38 -0500 Received: from [204.127.203.184] (helo=sccmmhc02.asp.att.net) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AmroI-0004zl-RK; Sat, 31 Jan 2004 04:53:06 -0500 Received: from elma (12-222-155-71.client.insightbb.com[12.222.155.71]) by sccmmhc02.asp.att.net (sccmmhc02) with SMTP id <20040131095152mm200kcp07e>; Sat, 31 Jan 2004 09:51:52 +0000 From: Christian Grothoff Organization: Purdue University, Department of Computer Science To: info-gnunet@gnu.org, help-gnunet@gnu.org, gnunet-developers@gnu.org Date: Sat, 31 Jan 2004 04:52:52 -0500 User-Agent: KMail/1.5.4 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401310452.52312.grothoff@cs.purdue.edu> Cc: Subject: [GNUnet-developers] GNUnet 0.6.1b released X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Jan 2004 09:54:40 -0000 This release fixes various bugs, including inconsistencies with updateable SBlocks, improvements in the bandwidth allocation code (higher utilization with smaller spikes), fewer database accesses (size of bloomfilter was corrected), a routing bug causing very old queries to be forwarded (saves bandwidth) and a rare crash due to an integer overflow. Furthermore, indexed content is no longer overwritten by migrated content (saves space) and gnunet-gtk can now list gnunet-directories that have been stored locally (directory import). You must run gnunet-check -ra to migrate from 0.6.1a (or earlier) to 0.6.1b. Running gnunet-check -ra will fix the bloomfilter size, recover the indexed content and possibly fix other problems. As usual, the code is available at http://www.ovmj.org/GNUnet/download.php3 and on all GNU mirrors (once they synchronize). Note that RPMs are no longer provided (is someone is building RPMs and willing to share and support them, please let me know). Christian From MAILER-DAEMON Sat Jan 31 07:40:16 2004 Received: from list by monty-python.gnu.org with archive (Exim 4.24) id 1AmuQ3-0008Kn-Ta for mharc-gnunet-developers@gnu.org; Sat, 31 Jan 2004 07:40:15 -0500 Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AmuNs-0007vn-2v for gnunet-developers@gnu.org; Sat, 31 Jan 2004 07:38:00 -0500 Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AmuLi-0007bj-4H for gnunet-developers@gnu.org; Sat, 31 Jan 2004 07:36:18 -0500 Received: from [203.59.3.35] (helo=mail.iinet.net.au) by monty-python.gnu.org with smtp (Exim 4.24) id 1AmuJw-0007Uq-Fe for gnunet-developers@gnu.org; Sat, 31 Jan 2004 07:33:56 -0500 Received: (qmail 15581 invoked from network); 31 Jan 2004 12:30:19 -0000 Received: from unknown (HELO localhost) (203.217.35.128) by mail.iinet.net.au with SMTP; 31 Jan 2004 12:30:17 -0000 Date: Sat, 31 Jan 2004 23:31:11 +1100 From: Glenn McGrath To: gnunet-developers@gnu.org Subject: Re: [GNUnet-developers] GNUnet 0.6.1b released Message-Id: <20040131233111.6eea4c29.bug1@iinet.net.au> In-Reply-To: <200401310452.52312.grothoff@cs.purdue.edu> References: <200401310452.52312.grothoff@cs.purdue.edu> X-Mailer: Sylpheed version 0.9.8 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: gnunet-developers@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Mailinglist for GNUnet developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Jan 2004 12:40:13 -0000 On Sat, 31 Jan 2004 04:52:52 -0500 Christian Grothoff wrote: > As usual, the code is available at > > http://www.ovmj.org/GNUnet/download.php3 Ive uploaded new debian packages, ive set it up to automatically run gnunet-check -ra at isntall time. Glenn