[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[GNUnet-SVN] r211 - in GNUnet: contrib src/applications/datastore src/ap
|
From: |
grothoff |
|
Subject: |
[GNUnet-SVN] r211 - in GNUnet: contrib src/applications/datastore src/applications/fs src/applications/fs/ecrs src/applications/fs/fsui src/applications/fs/module src/applications/fs/tools src/applications/gap src/applications/sqstore_mysql src/applications/sqstore_sqlite src/include src/server |
|
Date: |
Mon, 7 Feb 2005 17:47:49 -0800 (PST) |
Author: grothoff
Date: 2005-02-07 17:47:48 -0800 (Mon, 07 Feb 2005)
New Revision: 211
Modified:
GNUnet/contrib/gnunet.root
GNUnet/src/applications/datastore/datastore.c
GNUnet/src/applications/datastore/filter.c
GNUnet/src/applications/datastore/prefetch.c
GNUnet/src/applications/fs/ecrs/unindex.c
GNUnet/src/applications/fs/ecrs/upload.c
GNUnet/src/applications/fs/ecrs_core.c
GNUnet/src/applications/fs/fsui/fsui.c
GNUnet/src/applications/fs/fsui/upload.c
GNUnet/src/applications/fs/module/fs.c
GNUnet/src/applications/fs/module/ondemand.c
GNUnet/src/applications/fs/module/ondemand.h
GNUnet/src/applications/fs/tools/gnunet-insert.c
GNUnet/src/applications/gap/gap.c
GNUnet/src/applications/sqstore_mysql/mysql.c
GNUnet/src/applications/sqstore_mysql/mysqltest.c
GNUnet/src/applications/sqstore_sqlite/sqlite.c
GNUnet/src/applications/sqstore_sqlite/sqlitetest.c
GNUnet/src/include/ecrs_core.h
GNUnet/src/server/tcpserver.c
GNUnet/src/server/version.c
Log:
various bugfixes
Modified: GNUnet/contrib/gnunet.root
===================================================================
--- GNUnet/contrib/gnunet.root 2005-02-07 23:33:03 UTC (rev 210)
+++ GNUnet/contrib/gnunet.root 2005-02-08 01:47:48 UTC (rev 211)
@@ -104,8 +104,10 @@
#
# If you do not specify a HOSTLISTURL, you must copy valid hostkeys to
# data/hosts manually.
-# Default: HOSTLISTURL = "http://www.ovmj.org/GNUnet/download/hostlist
http://www.woodtick.co.uk/hostlist";
-HOSTLISTURL = "http://www.ovmj.org/GNUnet/download/hostlist
http://gnunet.wensley.org.uk/hostlist";
+# Default: HOSTLISTURL = "http://ovmj.org/GNUnet/download/hostlist
http://www.woodtick.co.uk/hostlist";
+# The above are currently hostlists for 0.6.x, we'll have to have new ones for
+# 0.7.x -- once we're a bit further along.
+# HOSTLISTURL = "http://gnunet.org/hostlist"; # not yet on 0.7, will not yet
work
# If you have to use a proxy for outbound HTTP connections,
# specify the proxy configuration here. Default is no proxy.
@@ -137,9 +139,8 @@
# Using the fs tools for downloading may require the traffic module
# to be loaded!
#
-# Typical choices are: ""
-# Default: APPLICATIONS = ""
-APPLICATIONS = "advertising topology"
+# Default: APPLICATIONS = "advertising topology fs getoption stats traffic"
+APPLICATIONS = "advertising topology fs getoption stats traffic"
# Which transport mechanisms are available? Use space-separated list
# of the modules, e.g. "udp smtp tcp". The order is irrelevant, each
@@ -481,9 +482,9 @@
################################################
# Options for anonymous filesharing (AFS).
################################################
-[AFS]
+[FS]
-# How much disk space (MB) is GNUnet allowed to use for anonymous file
+# How much disk space (MB) is GNUnet allowed to use for file
# sharing? This does not take indexed files into account, only the
# space directly used by GNUnet is accounted for. GNUnet will gather
# content from the network if the current space-consumption is below
@@ -493,52 +494,8 @@
# Note that if you change the quota, you need to run gnunet-convert,
# otherwise your databases will be inconsistent and gnunetd will
# refuse to work. Default is 1024 (1 GB)
-DISKQUOTA = 1024
+QUOTA = 1024
-# What degree of receiver anonymity is required? If set to 0, GNUnet
-# will try to download the file as fast as possible without any
-# additional slowdown by the anonymity code. Note that you will still
-# have a fair degree of anonymity depending on the current network
-# load and the power of the adversary. The download is still unlikely
-# to be terribly fast since the sender may have requested
-# sender-anonymity and since in addition to that, GNUnet will still do
-# the anonymous routing.
-#
-# This option can be used to limit requests further than that. In
-# particular, you can require GNUnet to receive certain amounts of
-# traffic from other peers before sending your queries. This way, you
-# can gain very high levels of anonymity - at the expense of much more
-# traffic and much higher latency. So set it only if you really
-# believe you need it.
-#
-# The definition of ANONYMITY-RECEIVE is the following:
-# If the value v # is < 1000, it means that if GNUnet routes n bytes
-# of messages from # foreign peers, it may originate n/v bytes of
-# queries in the same # time-period. The time-period is twice the
-# average delay that GNUnet # deferrs forwarded queries.
-#
-# If the value v is >= 1000, it means that if GNUnet routes n bytes
-# of QUERIES from at least (v % 1000) peers, it may originate
-# n/v/1000 bytes of queries in the same time-period.
-#
-# The default is 0 and this should be fine for most users. Also notice
-# that if you choose values above 1000, you may end up having no
-# throughput at all, especially if many of your fellow GNUnet-peers do
-# the same.
-ANONYMITY-RECEIVE = 0
-
-# You can also request a certain degree of anonymity for the files and
-# blocks that you are sharing. In this case, only a certain faction of
-# the traffic that you are routing will be allowed to be replies that
-# originate from your machine. Again, 0 means unlimited.
-#
-# The semantics of ANONYMITY-SEND are equivalent to the semantics of
-# ANONYMITY-RECEIVE.
-#
-# The default is 0 and this should be fine for most users.
-ANONYMITY-SEND = 0
-
-
# Should we participate in content migration? If you say yes here,
# GNUnet will migrate content to your server, and you will not be able
# to control what data is stored on your machine. This option has
@@ -575,8 +532,8 @@
# Default is YES.
ACTIVEMIGRATION = YES
-# Where to store the AFS related data (content, etc)?
-AFSDIR = $GNUNETD_HOME/data/afs/
+# Where to store the FS related data (content, etc)?
+DIR = $GNUNETD_HOME/data/fs/
# Where to store indexed files (NEW!)
# Note that you MUST not copy files directly to this
@@ -607,7 +564,20 @@
INDEX-QUOTA = 8192
+#######################################
+# MySQL specific options.
+#######################################
+[MYSQL]
+
+# If you suffer from too slow index/insert speeds,
+# you might try to set this to YES for a small
+# efficiency boost. If you run into any trouble
+# because of it, you're on your own.
+# Default: NO
+DELAYED = NO
+
+
#######################################
# TESTBED (experimental!)
#######################################
@@ -633,7 +603,15 @@
# application port (default: 2087).
# LOGIN =
+########################################
+# GAP options
+########################################
+[GAP]
+# Size of the routing table.
+# Default: 65536
+TABLESIZE = 65536
+
########################################
# DHT (experimental)
########################################
Modified: GNUnet/src/applications/datastore/datastore.c
===================================================================
--- GNUnet/src/applications/datastore/datastore.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/datastore/datastore.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -246,7 +246,7 @@
*/
static void cronMaintenance(void * unused) {
long long tmpAvailable
- = getConfigurationInt("AFS", "QUOTA") * 1024 * 1024; /* MB to bytes */
+ = getConfigurationInt("FS", "QUOTA") * 1024 * 1024; /* MB to bytes */
available = tmpAvailable - sq->getSize();
if (available < MIN_FREE) {
sq->iterateExpirationTime(ANY_BLOCK,
@@ -276,8 +276,8 @@
return NULL;
}
quota
- = htonl(getConfigurationInt("AFS", "QUOTA"));
- stateWriteContent("AFS-LAST-QUOTA",
+ = htonl(getConfigurationInt("FS", "QUOTA"));
+ stateWriteContent("FS-LAST-QUOTA",
sizeof(int),
"a);
@@ -339,9 +339,9 @@
int lastQuota;
quota
- = getConfigurationInt("AFS", "QUOTA");
+ = getConfigurationInt("FS", "QUOTA");
lq = NULL;
- if (sizeof(int) != stateReadContent("AFS-LAST-QUOTA",
+ if (sizeof(int) != stateReadContent("FS-LAST-QUOTA",
(void**)&lq))
return; /* first start? */
lastQuota = ntohl(*lq);
Modified: GNUnet/src/applications/datastore/filter.c
===================================================================
--- GNUnet/src/applications/datastore/filter.c 2005-02-07 23:33:03 UTC (rev
210)
+++ GNUnet/src/applications/datastore/filter.c 2005-02-08 01:47:48 UTC (rev
211)
@@ -37,10 +37,10 @@
char * fn;
char * bf;
- fn = getFileName("AFS",
- "AFSDIR",
+ fn = getFileName("FS",
+ "DIR",
_("Configuration must specify directory for "
- "AFS data in section '%s' under '%s'.\n"));
+ "FS data in section '%s' under '%s'.\n"));
mkdirp(fn);
bf = MALLOC(strlen(fn)+
strlen("/bloomfilter")+1);
@@ -60,8 +60,8 @@
/* read existing quota, check if it changed */
qt = NULL;
- quota = getConfigurationInt("AFS",
- "DISKQUOTA") * 1024;
+ quota = getConfigurationInt("FS",
+ "QUOTA") * 1024;
bf_size = quota/8; /* 8 bit per entry, 1 bit per kb in DB */
filter
Modified: GNUnet/src/applications/datastore/prefetch.c
===================================================================
--- GNUnet/src/applications/datastore/prefetch.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/datastore/prefetch.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -109,10 +109,21 @@
* Acquire new block(s) to the migration buffer.
*/
static void * rcbAcquire(void * unused) {
- while (doneSignal == NULL)
+ int load;
+ while (doneSignal == NULL) {
sq->iterateExpirationTime(0,
&aquire,
NULL);
+ /* sleep here, too - otherwise we start looping immediately
+ if there is no content in the DB! */
+ load = getCPULoad();
+ if (load < 10)
+ load = 10; /* never sleep less than 500 ms */
+ if (load > 100)
+ load = 100; /* never sleep longer than 5 seconds since that
+ might show up badly in the shutdown sequence... */
+ gnunet_util_sleep(50 * cronMILLIS * load);
+ }
SEMAPHORE_UP(doneSignal);
return NULL;
}
Modified: GNUnet/src/applications/fs/ecrs/unindex.c
===================================================================
--- GNUnet/src/applications/fs/ecrs/unindex.c 2005-02-07 23:33:03 UTC (rev
210)
+++ GNUnet/src/applications/fs/ecrs/unindex.c 2005-02-08 01:47:48 UTC (rev
211)
@@ -58,20 +58,22 @@
unsigned int size;
unsigned int present;
Datastore_Value * value;
+ DBlock * db;
CHK ichk;
- size = ntohl(iblocks[level]->size) - sizeof(Datastore_Value);
+ size = ntohl(iblocks[level]->size) - sizeof(Datastore_Value) -
sizeof(DBlock);
present = size / sizeof(CHK);
+ db = (DBlock*) &iblocks[level][1];
if (present == CHK_PER_INODE) {
- fileBlockGetKey((char*) &iblocks[level][1],
+ fileBlockGetKey((char*) db,
size,
&ichk.key);
- fileBlockGetQuery((char*) &iblocks[level][1],
+ fileBlockGetQuery((char*) db,
size,
&ichk.query);
if (OK != pushBlock(sock, &ichk, level+1, iblocks))
return SYSERR;
- fileBlockEncode((char*) &iblocks[level][1],
+ fileBlockEncode(db,
size,
&ichk.query,
&value);
@@ -81,10 +83,10 @@
return SYSERR;
}
FREE(value);
- size = 0;
+ size = sizeof(DBlock);
}
/* append CHK */
- memcpy(&((char*)&iblocks[level][1])[size],
+ memcpy(&((char*)db)[size],
chk,
sizeof(CHK));
iblocks[level]->size = htonl(size + sizeof(Datastore_Value));
@@ -124,7 +126,7 @@
#endif
serverDir
= getConfigurationOptionValue(sock,
- "AFS",
+ "FS",
"INDEX-DIRECTORY");
if (serverDir == NULL)
return OK;
@@ -217,6 +219,7 @@
unsigned int size;
Datastore_Value ** iblocks;
Datastore_Value * dblock;
+ DBlock * db;
Datastore_Value * value;
GNUNET_TCP_SOCKET * sock;
HashCode160 fileId;
@@ -285,20 +288,23 @@
LOG_FILE_STRERROR(LOG_WARNING, "OPEN", filename);
return SYSERR;
}
- dblock = MALLOC(sizeof(Datastore_Value) + DBLOCK_SIZE);
- dblock->size = htonl(sizeof(Datastore_Value) + DBLOCK_SIZE);
+ dblock = MALLOC(sizeof(Datastore_Value) + DBLOCK_SIZE + sizeof(DBlock));
+ dblock->size = htonl(sizeof(Datastore_Value) + DBLOCK_SIZE + sizeof(DBlock));
dblock->anonymityLevel = htonl(0);
dblock->prio = htonl(0);
dblock->type = htonl(D_BLOCK);
dblock->expirationTime = htonll(0);
+ db = (DBlock*) &dblock[1];
+ db->type = htonl(D_BLOCK);
iblocks = MALLOC(sizeof(Datastore_Value*) * treedepth);
for (i=0;i<treedepth;i++) {
- iblocks[i] = MALLOC(sizeof(Datastore_Value) + IBLOCK_SIZE);
- iblocks[i]->size = htonl(sizeof(Datastore_Value));
+ iblocks[i] = MALLOC(sizeof(Datastore_Value) + IBLOCK_SIZE +
sizeof(DBlock));
+ iblocks[i]->size = htonl(sizeof(Datastore_Value) + sizeof(DBlock));
iblocks[i]->anonymityLevel = htonl(0);
iblocks[i]->prio = htonl(0);
iblocks[i]->type = htonl(D_BLOCK);
iblocks[i]->expirationTime = htonll(0);
+ ((DBlock*) &iblocks[i][1])->type = htonl(D_BLOCK);
}
pos = 0;
@@ -311,13 +317,19 @@
size = DBLOCK_SIZE;
if (size > filesize - pos) {
size = filesize - pos;
- memset(&dblock[1], 0, DBLOCK_SIZE);
+ memset(&db[1],
+ 0,
+ DBLOCK_SIZE);
}
- if (size != READ(fd, &dblock[1], size)) {
- LOG_FILE_STRERROR(LOG_WARNING, "READ", filename);
+ if (size != READ(fd,
+ &db[1],
+ size)) {
+ LOG_FILE_STRERROR(LOG_WARNING,
+ "READ",
+ filename);
goto ERROR;
}
- size = DBLOCK_SIZE; /* padding! */
+ size = DBLOCK_SIZE + sizeof(DBlock); /* padding! */
if (tt != NULL)
if (OK != tt(ttClosure))
goto ERROR;
@@ -333,7 +345,7 @@
iblocks))
goto ERROR;
if (! wasIndexed) {
- fileBlockEncode((char*) &dblock[1],
+ fileBlockEncode(db,
size,
&chk.query,
&value);
@@ -356,10 +368,11 @@
goto ERROR;
for (i=0;i<treedepth;i++) {
size = ntohl(iblocks[i]->size) - sizeof(Datastore_Value);
- fileBlockGetKey((char*) &iblocks[i],
+ db = (DBlock*) &iblocks[i];
+ fileBlockGetKey((char*) db,
size,
&chk.key);
- fileBlockGetQuery((char*) &iblocks[i],
+ fileBlockGetQuery((char*) db,
size,
&chk.query);
if (OK != pushBlock(sock,
@@ -367,7 +380,7 @@
i+1,
iblocks))
goto ERROR;
- fileBlockEncode((char*) &iblocks[i][1],
+ fileBlockEncode(db,
size,
&chk.query,
&value);
Modified: GNUnet/src/applications/fs/ecrs/upload.c
===================================================================
--- GNUnet/src/applications/fs/ecrs/upload.c 2005-02-07 23:33:03 UTC (rev
210)
+++ GNUnet/src/applications/fs/ecrs/upload.c 2005-02-08 01:47:48 UTC (rev
211)
@@ -51,20 +51,22 @@
unsigned int size;
unsigned int present;
Datastore_Value * value;
+ DBlock * db;
CHK ichk;
- size = ntohl(iblocks[level]->size) - sizeof(Datastore_Value);
+ size = ntohl(iblocks[level]->size) - sizeof(Datastore_Value) -
sizeof(DBlock);
present = size / sizeof(CHK);
+ db = (DBlock*) &iblocks[level][1];
if (present == CHK_PER_INODE) {
- fileBlockGetKey((char*) &iblocks[level][1],
+ fileBlockGetKey((char*) db,
size,
&ichk.key);
- fileBlockGetQuery((char*) &iblocks[level][1],
+ fileBlockGetQuery((char*) db,
size,
&ichk.query);
if (OK != pushBlock(sock, &ichk, level+1, iblocks))
return SYSERR;
- fileBlockEncode((char*) &iblocks[level][1],
+ fileBlockEncode(db,
size,
&ichk.query,
&value);
@@ -74,13 +76,13 @@
return SYSERR;
}
FREE(value);
- size = 0;
+ size = sizeof(DBlock); /* type */
}
/* append CHK */
- memcpy(&((char*)&iblocks[level][1])[size],
+ memcpy(&((char*)db)[size],
chk,
sizeof(CHK));
- iblocks[level]->size = htonl(size + sizeof(Datastore_Value));
+ iblocks[level]->size = htonl(size + sizeof(Datastore_Value) +
sizeof(DBlock));
return OK;
}
@@ -107,7 +109,7 @@
return;
serverDir
= getConfigurationOptionValue(sock,
- "AFS",
+ "FS",
"INDEX-DIRECTORY");
if (serverDir == NULL)
return;
@@ -198,6 +200,7 @@
unsigned int size;
Datastore_Value ** iblocks;
Datastore_Value * dblock;
+ DBlock * db;
Datastore_Value * value;
GNUNET_TCP_SOCKET * sock;
HashCode160 fileId;
@@ -251,20 +254,23 @@
LOG_FILE_STRERROR(LOG_WARNING, "OPEN", filename);
return SYSERR;
}
- dblock = MALLOC(sizeof(Datastore_Value) + DBLOCK_SIZE);
- dblock->size = htonl(sizeof(Datastore_Value) + DBLOCK_SIZE);
+ dblock = MALLOC(sizeof(Datastore_Value) + DBLOCK_SIZE + sizeof(DBlock));
+ dblock->size = htonl(sizeof(Datastore_Value) + DBLOCK_SIZE + sizeof(DBlock));
dblock->anonymityLevel = htonl(anonymityLevel);
dblock->prio = htonl(priority);
dblock->type = htonl(D_BLOCK);
dblock->expirationTime = htonll(expirationTime);
+ db = (DBlock*) &dblock[1];
+ db->type = htonl(D_BLOCK);
iblocks = MALLOC(sizeof(Datastore_Value*) * treedepth);
for (i=0;i<treedepth;i++) {
- iblocks[i] = MALLOC(sizeof(Datastore_Value) + IBLOCK_SIZE);
- iblocks[i]->size = htonl(sizeof(Datastore_Value));
+ iblocks[i] = MALLOC(sizeof(Datastore_Value) + IBLOCK_SIZE +
sizeof(DBlock));
+ iblocks[i]->size = htonl(sizeof(Datastore_Value) + sizeof(DBlock));
iblocks[i]->anonymityLevel = htonl(anonymityLevel);
iblocks[i]->prio = htonl(priority);
iblocks[i]->type = htonl(D_BLOCK);
iblocks[i]->expirationTime = htonll(expirationTime);
+ ((DBlock*) &iblocks[i][1])->type = htonl(D_BLOCK);
}
pos = 0;
@@ -277,9 +283,14 @@
size = DBLOCK_SIZE;
if (size > filesize - pos) {
size = filesize - pos;
- memset(&dblock[1], 0, DBLOCK_SIZE);
+ memset(&db[1],
+ 0,
+ DBLOCK_SIZE);
}
- if (size != READ(fd, &dblock[1], size)) {
+ dblock->size = htonl(sizeof(Datastore_Value) + size + sizeof(DBlock));
+ if (size != READ(fd,
+ &db[1],
+ size)) {
LOG_FILE_STRERROR(LOG_WARNING, "READ", filename);
goto ERROR;
}
@@ -299,19 +310,19 @@
iblocks))
goto ERROR;
if (doIndex) {
- if (OK != FS_index(sock,
- &fileId,
- dblock,
- pos))
+ if (SYSERR == FS_index(sock,
+ &fileId,
+ dblock,
+ pos))
goto ERROR;
} else {
- fileBlockEncode((char*) &dblock[1],
+ fileBlockEncode(db,
size,
&chk.query,
&value);
*value = *dblock; /* copy options! */
- if (OK != FS_insert(sock,
- value)) {
+ if (SYSERR == FS_insert(sock,
+ value)) {
FREE(value);
goto ERROR;
}
@@ -330,10 +341,13 @@
goto ERROR;
for (i=0;i<treedepth;i++) {
size = ntohl(iblocks[i]->size) - sizeof(Datastore_Value);
- fileBlockGetKey((char*) &iblocks[i],
+ if (size == sizeof(DBlock))
+ continue;
+ db = (DBlock*) &iblocks[i];
+ fileBlockGetKey((char*) db,
size,
&chk.key);
- fileBlockGetQuery((char*) &iblocks[i],
+ fileBlockGetQuery((char*) db,
size,
&chk.query);
if (OK != pushBlock(sock,
@@ -341,7 +355,7 @@
i+1,
iblocks))
goto ERROR;
- fileBlockEncode((char*) &iblocks[i][1],
+ fileBlockEncode(db,
size,
&chk.query,
&value);
Modified: GNUnet/src/applications/fs/ecrs_core.c
===================================================================
--- GNUnet/src/applications/fs/ecrs_core.c 2005-02-07 23:33:03 UTC (rev
210)
+++ GNUnet/src/applications/fs/ecrs_core.c 2005-02-08 01:47:48 UTC (rev
211)
@@ -42,7 +42,7 @@
* @return OK on success, SYSERR if data does not
* match the query
*/
-int fileBlockEncode(const char * data,
+int fileBlockEncode(const DBlock * data,
unsigned int len,
const HashCode160 * query,
Datastore_Value ** value) {
@@ -50,7 +50,9 @@
SESSIONKEY skey;
unsigned char iv[BLOWFISH_BLOCK_LENGTH]; /* initial value */
Datastore_Value * val;
+ DBlock * db;
+ GNUNET_ASSERT(len > sizeof(DBlock));
GNUNET_ASSERT((data!=NULL) && (query != NULL));
hash(data, len, &hc);
hashToKey(&hc,
@@ -64,12 +66,13 @@
val->prio = htonl(0);
val->anonymityLevel = htonl(0);
val->expirationTime = htonl(0);
- ((DBlock*) &val[1])->type = htonl(D_BLOCK);
- GNUNET_ASSERT(len == encryptBlock(data,
- len,
+ db = (DBlock*) &val[1];
+ db->type = htonl(D_BLOCK);
+ GNUNET_ASSERT(len == encryptBlock(&data[1],
+ len - sizeof(DBlock),
&skey,
iv,
- &val[1]));
+ &db[1]));
hash(val,
len,
&hc);
@@ -79,6 +82,7 @@
return OK;
} else {
FREE(val);
+ BREAK();
*value = NULL;
return SYSERR;
}
@@ -91,7 +95,10 @@
void fileBlockGetKey(const char * data,
unsigned int len,
HashCode160 * key) {
- hash(data, len, key);
+ GNUNET_ASSERT(len >= sizeof(unsigned int));
+ hash(&data[sizeof(unsigned int)],
+ len - sizeof(unsigned int),
+ key);
}
/**
@@ -106,6 +113,9 @@
SESSIONKEY skey;
unsigned char iv[BLOWFISH_BLOCK_LENGTH];
+ GNUNET_ASSERT(len >= sizeof(unsigned int));
+ data = &data[sizeof(unsigned int)];
+ len -= sizeof(unsigned int);
hash(data, len, &hc);
hashToKey(&hc,
&skey,
@@ -122,7 +132,10 @@
unsigned int getTypeOfBlock(unsigned int size,
const void * data) {
- GNUNET_ASSERT(size > 4);
+ if (size <= 4) {
+ BREAK();
+ return ANY_BLOCK; /* signal error */
+ }
return *((const unsigned int*)data);
}
@@ -141,9 +154,11 @@
unsigned int type;
type = getTypeOfBlock(size, data);
+ if (type == ANY_BLOCK)
+ return SYSERR;
switch (type) {
case D_BLOCK:
- hash(data, size, query);
+ fileBlockGetKey(data, size, query);
return OK;
case S_BLOCK: {
SBlock * sb;
Modified: GNUnet/src/applications/fs/fsui/fsui.c
===================================================================
--- GNUnet/src/applications/fs/fsui/fsui.c 2005-02-07 23:33:03 UTC (rev
210)
+++ GNUnet/src/applications/fs/fsui/fsui.c 2005-02-08 01:47:48 UTC (rev
211)
@@ -43,8 +43,10 @@
ret = MALLOC(sizeof(FSUI_Context));
memset(ret, 0, sizeof(FSUI_Context));
- gh = getConfigurationString("",
+ fn = getConfigurationString("",
"GNUNET_HOME");
+ gh = expandFileName(fn);
+ FREE(fn);
fn = MALLOC(strlen(gh) + strlen("fsui-lock") + 2);
strcpy(fn, gh);
FREE(gh);
Modified: GNUnet/src/applications/fs/fsui/upload.c
===================================================================
--- GNUnet/src/applications/fs/fsui/upload.c 2005-02-07 23:33:03 UTC (rev
210)
+++ GNUnet/src/applications/fs/fsui/upload.c 2005-02-08 01:47:48 UTC (rev
211)
@@ -446,8 +446,8 @@
utc = MALLOC(sizeof(UploadThreadClosure));
utc->anonymityLevel = ctx->anonymityLevel;
- utc->priority = getConfigurationInt("AFS",
- "UPLOAD-PRIORITY");
+ utc->priority = getConfigurationInt("FS",
+ "INSERT-PRIORITY");
utc->expiration = cronTime(NULL) + 120 * cronYEARS;
utc->ctx = ctx;
utc->isRecursive = NO;
@@ -513,8 +513,8 @@
utc->ctx = ctx;
utc->isRecursive = YES;
utc->anonymityLevel = ctx->anonymityLevel;
- utc->priority = getConfigurationInt("AFS",
- "UPLOAD-PRIORITY");
+ utc->priority = getConfigurationInt("FS",
+ "INSERT-PRIORITY");
utc->expiration = cronTime(NULL) + 120 * cronYEARS;
utc->extractors = EXTRACTOR_loadConfigLibraries(NULL, extractorPluginNames);
utc->globalUri = FSUI_parseArgvKeywordURI(globalKeywordCount,
Modified: GNUnet/src/applications/fs/module/fs.c
===================================================================
--- GNUnet/src/applications/fs/module/fs.c 2005-02-07 23:33:03 UTC (rev
210)
+++ GNUnet/src/applications/fs/module/fs.c 2005-02-08 01:47:48 UTC (rev
211)
@@ -248,18 +248,18 @@
HashCode160 query;
unsigned int type;
- if (ntohs(req->size) < sizeof(RequestIndex)) {
+ if (ntohs(req->size) < sizeof(RequestInsert)) {
BREAK();
return SYSERR;
}
ri = (RequestInsert*) req;
datum = MALLOC(sizeof(Datastore_Value) +
- ntohs(req->size) - sizeof(RequestIndex));
+ ntohs(req->size) - sizeof(RequestInsert));
datum->expirationTime = ri->expiration;
datum->prio = ri->prio;
datum->anonymityLevel = ri->anonymityLevel;
if (OK != getQueryFor(ntohs(ri->header.size) - sizeof(RequestInsert),
- (char*)&ri[1],
+ (const char*)&ri[1],
&query)) {
BREAK();
FREE(datum);
@@ -336,7 +336,7 @@
ntohl(ri->anonymityLevel),
&ri->fileId,
ntohs(ri->header.size) - sizeof(RequestIndex),
- (const char*) &ri[1]);
+ (const DBlock*) &ri[1]);
LOG(LOG_DEBUG,
"Sending confirmation of index request to client\n");
return coreAPI->sendValueToClient(sock,
@@ -769,11 +769,11 @@
hash("GNUNET_FS",
strlen("GNUNET_FS"),
&dht_table);
- if (getConfigurationInt("AFS",
- "DISKQUOTA") <= 0) {
+ if (getConfigurationInt("FS",
+ "QUOTA") <= 0) {
LOG(LOG_ERROR,
_("You must specify a postive number for '%s' in the configuration in
section '%s'.\n"),
- "DISKQUOTA", "AFS");
+ "QUOTA", "FS");
return SYSERR;
}
datastore = capi->requestService("datastore");
Modified: GNUnet/src/applications/fs/module/ondemand.c
===================================================================
--- GNUnet/src/applications/fs/module/ondemand.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/fs/module/ondemand.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -67,9 +67,9 @@
char * fn;
char * dir;
- dir = getFileName("AFS",
+ dir = getFileName("FS",
"INDEX-DIRECTORY",
- _("You must specify a directory for AFS files in the"
+ _("You must specify a directory for FS files in the"
" configuration in section '%s' under '%s'."));
mkdirp(dir); /* just in case */
hash2enc(fileId,
@@ -117,29 +117,33 @@
unsigned int anonymityLevel,
const HashCode160 * fileId,
unsigned int size,
- const char * content) {
+ const DBlock * content) {
char * fn;
int fd;
int ret;
OnDemandBlock odb;
HashCode160 key;
+ if (size <= sizeof(DBlock)) {
+ BREAK();
+ return SYSERR;
+ }
fn = getOnDemandFile(fileId);
fd = OPEN(fn,
O_CREAT|O_WRONLY,
S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); /* 644 */
if(fd == -1) {
LOG_FILE_STRERROR(LOG_ERROR, "open", fn);
- FREE(fn);
+ FREE(fn);
return SYSERR;
}
lseek(fd,
fileOffset,
SEEK_SET);
ret = WRITE(fd,
- content,
- size);
- if (ret == size) {
+ &content[1],
+ size - sizeof(DBlock));
+ if (ret == size - sizeof(DBlock)) {
ret = OK;
} else {
LOG_FILE_STRERROR(LOG_ERROR, "write", fn);
@@ -147,6 +151,8 @@
}
CLOSE(fd);
FREE(fn);
+ if (ret == SYSERR)
+ return ret;
odb.header.size = htonl(sizeof(OnDemandBlock));
odb.header.type = htonl(ONDEMAND_BLOCK);
@@ -154,7 +160,7 @@
odb.header.anonymityLevel = htonl(anonymityLevel);
odb.header.expirationTime = htonll(expiration);
odb.fileOffset = htonll(fileOffset);
- odb.blockSize = htonl(size);
+ odb.blockSize = htonl(size - sizeof(DBlock));
odb.fileId = *fileId;
/* compute the primary key */
fileBlockGetQuery(content,
@@ -164,13 +170,12 @@
ONDEMAND_BLOCK,
&checkPresent,
&odb.header);
- if (ret == SYSERR) {
+ if (ret <= 0) {
ret = datastore->put(&key,
&odb.header);
- if (ret != YES)
- ret = SYSERR; /* not stored */
- } else
+ } else {
ret = NO; /* already present! */
+ }
return ret;
}
@@ -193,6 +198,7 @@
int fileHandle;
int ret;
OnDemandBlock * odb;
+ DBlock * db;
if (ntohl(dbv->size) != sizeof(OnDemandBlock)) {
BREAK();
@@ -215,10 +221,10 @@
char * scratch;
int n;
- afsDir = getFileName("AFS",
- "AFSDIR",
+ afsDir = getFileName("FS",
+ "DIR",
_("Configuration file must specify directory for"
- " storage of AFS data in section '%s'"
+ " storage of FS data in section '%s'"
" under '%s'.\n"));
n = strlen(afsDir)+strlen(TRACKFILE)+8;
scratch = MALLOC(n);
@@ -243,22 +249,24 @@
CLOSE(fileHandle);
goto ERROR;
}
- iobuf = MALLOC(ntohl(odb->blockSize));
+ db = MALLOC(sizeof(DBlock) + ntohl(odb->blockSize));
+ db->type = htonl(D_BLOCK);
+ iobuf = (char*) &db[1];
blen = READ(fileHandle,
iobuf,
ntohl(odb->blockSize));
if (blen != ntohl(odb->blockSize)) {
LOG_FILE_STRERROR(LOG_ERROR, "read", fn);
FREE(fn);
- FREE(iobuf);
+ FREE(db);
CLOSE(fileHandle);
goto ERROR;
}
- ret = fileBlockEncode(iobuf,
- ntohl(odb->blockSize),
+ ret = fileBlockEncode(db,
+ ntohl(odb->blockSize) + sizeof(DBlock),
query,
enc);
- FREE(iobuf);
+ FREE(db);
FREE(fn);
if (ret == SYSERR)
goto ERROR;
Modified: GNUnet/src/applications/fs/module/ondemand.h
===================================================================
--- GNUnet/src/applications/fs/module/ondemand.h 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/fs/module/ondemand.h 2005-02-08 01:47:48 UTC
(rev 211)
@@ -40,7 +40,7 @@
unsigned int anonymityLevel,
const HashCode160 * fileId,
unsigned int size,
- const char * content);
+ const DBlock * content);
/**
* A query on the datastore resulted in the on-demand
Modified: GNUnet/src/applications/fs/tools/gnunet-insert.c
===================================================================
--- GNUnet/src/applications/fs/tools/gnunet-insert.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/fs/tools/gnunet-insert.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -124,7 +124,7 @@
case upload_progress:
if (*verboselevel == YES) {
delta = event->data.UploadProgress.eta - cronTime(NULL);
- printf(_("%16llu of %16llu bytes inserted (estimating %llu seconds to
completion)"),
+ printf(_("%16llu of %16llu bytes inserted (estimating %llu seconds to
completion) "),
event->data.UploadProgress.main_completed,
event->data.UploadProgress.main_total,
delta / cronSECONDS);
@@ -133,23 +133,23 @@
break;
case upload_complete:
if (*verboselevel == YES) {
- delta = event->data.UploadProgress.eta -
event->data.UploadProgress.start_time;
+ delta = event->data.UploadComplete.eta -
event->data.UploadComplete.start_time;
printf(_("\nUpload of '%s' complete, %llu bytes took %llu seconds (%8.3f
kbps).\n"),
- event->data.UploadProgress.filename,
- event->data.UploadProgress.main_total,
+ event->data.UploadComplete.filename,
+ event->data.UploadComplete.total,
delta / cronSECONDS,
(delta == 0)
? (double) (-1.0)
- : (double) (event->data.UploadProgress.main_total / 1024.0 *
cronSECONDS / delta));
+ : (double) (event->data.UploadComplete.total / 1024.0 *
cronSECONDS / delta));
}
fstring = ECRS_uriToString(event->data.UploadComplete.uri);
printf(_("File '%s' has URI: %s\n"),
event->data.UploadComplete.filename,
fstring);
FREE(fstring);
- if (0 == strcmp(event->data.DownloadProgress.main_filename,
- event->data.DownloadProgress.filename)) {
- postProcess(event->data.DownloadProgress.main_uri);
+ if (0 == strcmp(event->data.UploadComplete.main_filename,
+ event->data.UploadComplete.filename)) {
+ postProcess(event->data.UploadComplete.uri);
SEMAPHORE_UP(exitSignal);
}
Modified: GNUnet/src/applications/gap/gap.c
===================================================================
--- GNUnet/src/applications/gap/gap.c 2005-02-07 23:33:03 UTC (rev 210)
+++ GNUnet/src/applications/gap/gap.c 2005-02-08 01:47:48 UTC (rev 211)
@@ -1880,8 +1880,8 @@
}
random_qsel = randomi(0xFFFF);
indirectionTableSize =
- getConfigurationInt("AFS",
- "INDIRECTIONTABLESIZE");
+ getConfigurationInt("GAP",
+ "TABLESIZE");
if (indirectionTableSize < MIN_INDIRECTION_TABLE_SIZE)
indirectionTableSize = MIN_INDIRECTION_TABLE_SIZE;
ROUTING_indTable_
Modified: GNUnet/src/applications/sqstore_mysql/mysql.c
===================================================================
--- GNUnet/src/applications/sqstore_mysql/mysql.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/sqstore_mysql/mysql.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -120,8 +120,8 @@
* If you suffer from too slow index/insert speeds,
* you might try to define /etc/gnunet.conf option
*
- * [AFS]
- * MYSQL_DELAYED = YES
+ * [MYSQL]
+ * DELAYED = YES
*
* for small efficiency boost. The option will let MySQL bundle multiple
* inserts before actually writing them to disk. You shouldn't use this
@@ -182,11 +182,12 @@
*
*/
static Datastore_Datum * assembleDatum(MYSQL_ROW sql_row) {
-
Datastore_Datum * datum;
int contentSize;
contentSize = atol(sql_row[0]) - sizeof(Datastore_Value);
+ if (contentSize < 0)
+ return NULL; /* error */
datum = MALLOC(sizeof(Datastore_Datum) + contentSize);
datum->value.size = htonl(contentSize + sizeof(Datastore_Value));
@@ -203,7 +204,6 @@
memcpy(&datum[1],
sql_row[6],
contentSize);
-
return(datum);
}
@@ -248,11 +248,11 @@
* Close the database connection.
*/
static int iclose(mysqlHandle * dbhI) {
- if (dbh->dbf == NULL)
+ if (dbhI->dbf == NULL)
return SYSERR;
- MUTEX_DESTROY(&dbh->DATABASE_Lock_);
- mysql_close(dbh->dbf);
- dbh->dbf = NULL;
+ MUTEX_DESTROY(&dbhI->DATABASE_Lock_);
+ mysql_close(dbhI->dbf);
+ dbhI->dbf = NULL;
return OK;
}
@@ -303,18 +303,24 @@
FREE(scratch);
if (mysql_error(dbhI.dbf)[0]) {
LOG_MYSQL(LOG_ERROR, "mysql_query", &dbhI);
- MUTEX_UNLOCK(&dbhI.DATABASE_Lock_);
+ MUTEX_UNLOCK(&dbhI.DATABASE_Lock_);
+ iclose(&dbhI);
return(SYSERR);
}
if (!(sql_res=mysql_use_result(dbhI.dbf))) {
MUTEX_UNLOCK(&dbhI.DATABASE_Lock_);
+ iclose(&dbhI);
return(SYSERR);
}
- while ((sql_row=mysql_fetch_row(sql_res))) {
-
+ while ((sql_row=mysql_fetch_row(sql_res))) {
datum = assembleDatum(sql_row);
+ if (datum == NULL) {
+ LOG(LOG_WARNING,
+ _("Invalid data in MySQL database. Please verify integrity!\n"));
+ continue;
+ }
if( SYSERR == iter(&datum->key, &datum->value, closure) ) {
count = SYSERR;
FREE(datum);
@@ -359,9 +365,7 @@
if (OK != iopen(&dbhI))
return SYSERR;
-
MUTEX_LOCK(&dbhI.DATABASE_Lock_);
-
if(type==0) {
typestr[0]=0;
} else {
@@ -383,26 +387,31 @@
if (mysql_error(dbhI.dbf)[0]) {
LOG_MYSQL(LOG_ERROR, "mysql_query", &dbhI);
MUTEX_UNLOCK(&dbhI.DATABASE_Lock_);
+ iclose(&dbhI);
return(SYSERR);
}
if (!(sql_res=mysql_use_result(dbhI.dbf))) {
MUTEX_UNLOCK(&dbhI.DATABASE_Lock_);
+ iclose(&dbhI);
return(SYSERR);
}
while ((sql_row=mysql_fetch_row(sql_res))) {
datum = assembleDatum(sql_row);
+ if (datum == NULL) {
+ LOG(LOG_WARNING,
+ _("Invalid data in MySQL database. Please verify integrity!\n"));
+ continue;
+ }
if (SYSERR == iter(&datum->key, &datum->value, closure) ) {
count = SYSERR;
FREE(datum);
break;
}
FREE(datum);
-
count++;
- }
-
+ }
mysql_free_result(sql_res);
MUTEX_UNLOCK(&dbhI.DATABASE_Lock_);
iclose(&dbhI);
@@ -439,7 +448,7 @@
mysql_escape_string(escapedHash,
(char *)query,
sizeof(HashCode160));
- if( type!=0) {
+ if (type!=0) {
SNPRINTF(scratch,
256,
"SELECT %s FROM gn070"
@@ -457,7 +466,7 @@
}
FREE(escapedHash);
} else { /* query is NULL */
- if(type==0) {
+ if (type==0) {
SNPRINTF(scratch,
256,
"SELECT %s FROM gn070",
@@ -497,6 +506,11 @@
Datastore_Datum * datum;
datum = assembleDatum(sql_row);
+ if (datum == NULL) {
+ LOG(LOG_WARNING,
+ _("Invalid data in MySQL database. Please verify integrity!\n"));
+ continue;
+ }
if( SYSERR == iter(&datum->key,&datum->value, closure) ) {
count = SYSERR;
FREE(datum);
@@ -620,7 +634,7 @@
escapedBlock
);
}
- mysql_query(dbh->dbf,scratch);
+ mysql_query(dbh->dbf, scratch);
FREE(escapedHash);
FREE(escapedBlock);
FREE(scratch);
@@ -830,8 +844,8 @@
dbh = MALLOC(sizeof(mysqlHandle));
dbh->cnffile = cnffile;
- if (testConfigurationString("AFS",
- "MYSQL_DELAYED",
+ if (testConfigurationString("MYSQL",
+ "DELAYED",
"YES"))
dbh->useDelayed = YES;
else
Modified: GNUnet/src/applications/sqstore_mysql/mysqltest.c
===================================================================
--- GNUnet/src/applications/sqstore_mysql/mysqltest.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/sqstore_mysql/mysqltest.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -136,8 +136,8 @@
FREENONNULL(setConfigurationString("FILES",
"gnunet.conf",
"/tmp/gnunet_test/gnunet.conf"));
- FREENONNULL(setConfigurationString("AFS",
- "AFSDIR",
+ FREENONNULL(setConfigurationString("FS",
+ "DIR",
TEST_DB));
return OK;
}
Modified: GNUnet/src/applications/sqstore_sqlite/sqlite.c
===================================================================
--- GNUnet/src/applications/sqstore_sqlite/sqlite.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/sqstore_sqlite/sqlite.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -742,10 +742,10 @@
dbh->indexed = 0;
dbh->lastSync = 0;
- afsdir = getFileName("AFS",
- "AFSDIR",
- _("Configuration file must specify directory for "
- "storing AFS data in section '%s' under '%s'.\n"));
+ afsdir = getFileName("FS",
+ "DIR",
+ _("Configuration file must specify directory for "
+ "storing FS data in section '%s' under '%s'.\n"));
dir = MALLOC(strlen(afsdir) + strlen(CONTENTDIR) + 2);
strcpy(dir, afsdir);
strcat(dir, "/");
Modified: GNUnet/src/applications/sqstore_sqlite/sqlitetest.c
===================================================================
--- GNUnet/src/applications/sqstore_sqlite/sqlitetest.c 2005-02-07 23:33:03 UTC
(rev 210)
+++ GNUnet/src/applications/sqstore_sqlite/sqlitetest.c 2005-02-08 01:47:48 UTC
(rev 211)
@@ -136,8 +136,8 @@
FREENONNULL(setConfigurationString("FILES",
"gnunet.conf",
"/tmp/gnunet_test/gnunet.conf"));
- FREENONNULL(setConfigurationString("AFS",
- "AFSDIR",
+ FREENONNULL(setConfigurationString("FS",
+ "DIR",
TEST_DB));
return OK;
}
Modified: GNUnet/src/include/ecrs_core.h
===================================================================
--- GNUnet/src/include/ecrs_core.h 2005-02-07 23:33:03 UTC (rev 210)
+++ GNUnet/src/include/ecrs_core.h 2005-02-08 01:47:48 UTC (rev 211)
@@ -161,7 +161,7 @@
* @return OK on success, SYSERR if data does not
* match the query
*/
-int fileBlockEncode(const char * data,
+int fileBlockEncode(const DBlock * data,
unsigned int len,
const HashCode160 * query,
Datastore_Value ** value);
Modified: GNUnet/src/server/tcpserver.c
===================================================================
--- GNUnet/src/server/tcpserver.c 2005-02-07 23:33:03 UTC (rev 210)
+++ GNUnet/src/server/tcpserver.c 2005-02-08 01:47:48 UTC (rev 211)
@@ -284,9 +284,13 @@
ptyp);
MUTEX_UNLOCK(&handlerlock);
return SYSERR;
- } else
- callback(sender,
- msg);
+ } else {
+ if (OK != callback(sender,
+ msg)) {
+ MUTEX_UNLOCK(&handlerlock);
+ return SYSERR;
+ }
+ }
MUTEX_UNLOCK(&handlerlock);
return OK;
}
Modified: GNUnet/src/server/version.c
===================================================================
--- GNUnet/src/server/version.c 2005-02-07 23:33:03 UTC (rev 210)
+++ GNUnet/src/server/version.c 2005-02-08 01:47:48 UTC (rev 211)
@@ -79,7 +79,7 @@
simple alternative would be to require gnunet-update for any
configuration change, but that again would be too strict. */
dyncat(&string, "GNUNETD", "APPLICATIONS");
- dyncat(&string, "AFS", "QUOTA");
+ dyncat(&string, "FS", "QUOTA");
dyncat(&string, "MODULES", "sqstore");
hash(string,
strlen(string),
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [GNUnet-SVN] r211 - in GNUnet: contrib src/applications/datastore src/applications/fs src/applications/fs/ecrs src/applications/fs/fsui src/applications/fs/module src/applications/fs/tools src/applications/gap src/applications/sqstore_mysql src/applications/sqstore_sqlite src/include src/server,
grothoff <=