gnunet-developers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[GNUnet-developers] [Patch] configurable content store


From: Uli Luckas
Subject: [GNUnet-developers] [Patch] configurable content store
Date: Sat, 18 May 2002 01:14:02 +0200

Dear gnunet developers,
regarding the pros and cons of different content store approaches I
implemented a configurable content store. This gives developers the
chance to implement and experiment with new content stores easily and
without forking to far away from the main line of development.
The intent is to see many competing implementations in the future so we
have plenty of choice to choose the right one. (read: start coding your
ideas now :-)

The new content store model is based on Christian Grothoff's api to the
gdbm content store and encapsulates this api into an OO-alike model.
The encapsulation is done through a struct of function pointers,
pointing to the actual implementaiton. Implementation dependent data is
being held in a void pointer field of the struct in order to enable
multiple instances to be active simultaniously.

The struct is initialized through an implementation dependent
init_contentstore_XXX function call which is chosen, depending on a new
configuration option CONTENTDBTYPE which can be set to either "dir" or
"gdbm" in gnunet.conf to get one of the previous implementations.
For now the initialized struct is being held in a static variable and
abstracted away by a set of inline functions. Therefore the public api
needed to be changed only for the init_contentstore function.

Where to go from here?
1)
The next obvious step is content migration from one datastore to an
other.
By holding implementation dependent data in a generic (read: void*)
struct field and not in static variables we are able to instanciate
multiple content stores from one application and can therefore in a
matter of a few lines of code read all data from one content store and
write it to another.
This could be done from within gnunetd (gnunetd
--migreate-file=oldcontentstore --migrate-type=dir) or from a seperate
application (gnu-migrate --name=oldcontentstore --type=dir)

2)
Instead of initializing the content stores with a dir-/filename it might
be a good idea to initialize it with the HASH* config_ variable from
configure.c. This way each content store implementation can have
different configuration options.
On content store could for example split theit's content over N files to
avoid older systems 2Gig file limit. An option CONTENTFILES=5 would be
very handy for this store but wouldn't make any sense for the "dir"
content store.

Please feel free to coment on these ideas and on the patch itself.

Thanks
Uli
diff -Naur GNUnet/contrib/gnunet.conf GNUnet.plugin/contrib/gnunet.conf
--- GNUnet/contrib/gnunet.conf  Wed May  8 03:24:06 2002
+++ GNUnet.plugin/contrib/gnunet.conf   Fri May 17 19:23:15 2002
@@ -251,6 +251,11 @@
 # dedicated partition optimally.
 CONTENTDIR     = $GNUNET_HOME/data/content/
 
+# Gnunet can store your content in a flat file database
+# or in different database formats. Your choices, for
+# now are "dir" or "gdbm".
+CONTENTDBTYPE   = gdbm
+
 # Where to store the credit for all hosts?
 # Should not be shared with other nodes, but contains
 # no 'secret' data either. Malicious hosts should NOT
diff -Naur GNUnet/contrib/gnunet.conf.root 
GNUnet.plugin/contrib/gnunet.conf.root
--- GNUnet/contrib/gnunet.conf.root     Mon Mar 18 12:09:26 2002
+++ GNUnet.plugin/contrib/gnunet.conf.root      Fri May 17 19:23:11 2002
@@ -185,6 +185,11 @@
 # dedicated partition optimally.
 CONTENTDIR     = $GNUNET_HOME/data/content/
 
+# Gnunet can store your content in a flat file database
+# or in different database formats. Your choices, for
+# now are "dir" or "gdbm".
+CONTENTDBTYPE   = gdbm
+
 # Where to store the credit for all hosts?
 # Should not be shared with other nodes, but contains
 # no 'secret' data either. Malicious hosts should NOT
diff -Naur GNUnet/src/common/configuration.c 
GNUnet.plugin/src/common/configuration.c
--- GNUnet/src/common/configuration.c   Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/common/configuration.c    Fri May 17 19:18:36 2002
@@ -55,6 +55,7 @@
 #define FILES_COLLISION              "COLLISION"
 #define FILES_SHARED_LIST     "FILELIST"
 #define FILES_CONTENT_DIR     "CONTENTDIR"
+#define FILES_CONTENT_DBTYPE  "CONTENTDBTYPE"
 #define FILES_CREDIT_DIR      "CREDITDIR"
 #define FILES_HOSTS_DIR       "HOSTS"
 
@@ -256,6 +257,18 @@
     errexit("Configuration file must specify directory for credits in section 
%s under %s.\n",
            FILES_SECTION,
            FILES_CREDIT_DIR);
+  return f;
+}
+
+char * getDatabaseType() {
+  char * f 
+    = (char*) CONF_get_string(config_,
+                                FILES_SECTION,
+                                FILES_CONTENT_DBTYPE);
+  if (f == NULL) 
+    errexit("Configuration file must specify database type for content in 
section %s under %s.\n",
+           FILES_SECTION,
+           FILES_CONTENT_DBTYPE);
   return f;
 }
 
diff -Naur GNUnet/src/include/util/contentdatabase.h 
GNUnet.plugin/src/include/util/contentdatabase.h
--- GNUnet/src/include/util/contentdatabase.h   Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/include/util/contentdatabase.h    Fri May 17 23:34:58 2002
@@ -18,44 +18,99 @@
 */
 /**
  * ContentDatabse - management of the 1k blocks...
- * @author Christian Grothoff
- * @file gnet/include/contentdatabase.h
+ * @author Uli Luckas, Christian Grothoff
+ * @file gnet/include/storage.h
  **/
 
 #ifndef CONTENTDATABASE_H
 #define CONTENTDATABASE_H
 
 #include "config.h"
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <string.h>
+#include <openssl/ripemd.h>
+#ifndef SOMEBSD
+#include <wordexp.h>
+#endif
+#include <tdb.h>
+
 #include "storage.h"
+#include "contentdatabase_gdbm.h"
 
-/**
- * init the storage module.
- * @param dir the name of the directory/file
- *        containing the content database
- **/
-void initContentDatabase(char * dir);
+struct contentDatabase {
+  /**
+   * Clean shutdown of the storage module (not used at the moment)
+   **/
+  void (*doneContentDatabase)(struct contentDatabase *contentDB);
+  
+  /**
+   * Free space in the database by removing one file
+   * @param fn the name of the file (without directory)
+   **/
+  void (*unlinkFromDB)(struct contentDatabase *contentDB, 
+                      HashCode160 * name);
+ 
+  /**
+   * Read the contents of a bucket to a buffer.
+   * @param fileName the name of the file, not freed!
+   * @param len the maximum number of bytes to read
+   * @param result the buffer to write the result to
+   * @return the number of bytes read on success, -1 on failure
+   **/ 
+  int (*readContent)(struct contentDatabase *contentDB, 
+                    HashCode160 * name,
+                    int  len,
+                    void * result);
+
+  /**
+   * Write content to a file. Check for reduncancy and eventually
+   * append.
+   * @param fn the name of the file (without directory)
+   * @param block the CONTENT_SIZE bytes long block (no header!)
+   **/
+  void (*writeContent)(struct contentDatabase *contentDB, 
+                      HashCode160 * name,
+                      CONTENT_Block * block);
+
+  /**
+   * Call a method for each entry in the database and
+   * call the callback method on it. 
+   * @return the number of items stored in the content database
+   **/
+  int (*forEachEntryInDatabase)(struct contentDatabase *contentDB, 
+                               void (*callback)(HashCode160*));
+
+  void * privateData;
+
+}; // struct contentDatabase
+
+inline void initContentDatabase(char * dbType, char * dir);
 
 /**
  * Clean shutdown of the storage module (not used at the moment)
  **/
-void doneContentDatabase();
+inline void doneContentDatabase();
 
 /**
- * Free space in the database by removing one file
- * @param fn the name of the file (without directory)
+ * Call a method for each entry in the database and
+ * call the callback method on it. 
+ * @return the number of items stored in the content database
  **/
-void unlinkFromDB(HashCode160 * fn);
- 
+inline int forEachEntryInDatabase(void (*callback)(HashCode160*));
+
 /**
- * Read the contents of a bucket to a buffer.
+ * Read the contents of a bucket to a buffer. 
  * @param fileName the name of the file, not freed!
  * @param len the maximum number of bytes to read
  * @param result the buffer to write the result to
  * @return the number of bytes read on success, -1 on failure
  **/ 
-int readContent(HashCode160 * fn,
-               int  len,
-               void * result);
+inline int readContent(HashCode160 * name,
+                      int  len,
+                      void * result);
 
 /**
  * Write content to a file. Check for reduncancy and eventually
@@ -63,16 +118,16 @@
  * @param fn the name of the file (without directory)
  * @param block the CONTENT_SIZE bytes long block (no header!)
  **/
-void writeContent(HashCode160 * fn, 
-                 CONTENT_Block * block);
+inline void writeContent(HashCode160 * name, 
+                        CONTENT_Block * block);
 
 /**
- * Call a method for each entry in the database and
- * call the callback method on it. 
- * @return the number of items stored in the content database
+ * Free space in the database by removing one file
+ * @param fn the name of the file (without directory)
  **/
-int forEachEntryInDatabase(void (*callback)(HashCode160*));
- 
+inline void unlinkFromDB(HashCode160 * name);
+
 #endif
 
 /* end of contentdatabase.h */
+
diff -Naur GNUnet/src/include/util/contentdatabase_directory.h 
GNUnet.plugin/src/include/util/contentdatabase_directory.h
--- GNUnet/src/include/util/contentdatabase_directory.h Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/include/util/contentdatabase_directory.h  Fri May 17 
18:56:16 2002
@@ -42,18 +42,19 @@
  * @param dir the name of the directory/file
  *        containing the content database
  **/
-void initContentDatabase_Directory(char * dir);
+struct contentDatabase *initContentDatabase_DirDB(char * dir);
 
 /**
  * Clean shutdown of the storage module (not used at the moment)
  **/
-void doneContentDatabase_Directory();
+void doneContentDatabase_DirDB(struct contentDatabase *contentDB);
 
 /**
  * Free space in the database by removing one file
  * @param fn the name of the file (without directory)
  **/
-void unlinkFromDB_Directory(HashCode160 * fn);
+void unlinkFromDB_DirDB(struct contentDatabase *contentDB,
+                           HashCode160 * fn);
  
 /**
  * Read the contents of a bucket to a buffer.
@@ -62,9 +63,10 @@
  * @param result the buffer to write the result to
  * @return the number of bytes read on success, -1 on failure
  **/ 
-int readContent_Directory(HashCode160 * fn,
-                    int  len,
-                    void * result);
+int readContent_DirDB(struct contentDatabase *contentDB,
+                         HashCode160 * fn,
+                         int  len,
+                         void * result);
 
 /**
  * Write content to a file. Check for reduncancy and eventually
@@ -72,16 +74,18 @@
  * @param fn the name of the file (without directory)
  * @param block the CONTENT_SIZE bytes long block (no header!)
  **/
-void writeContent_Directory(HashCode160 * fn, 
-                      CONTENT_Block * block);
+void writeContent_DirDB(struct contentDatabase *contentDB,
+                           HashCode160 * fn, 
+                           CONTENT_Block * block);
 
 /**
  * Call a method for each entry in the database and
  * call the callback method on it. 
  * @return the number of items stored in the content database
  **/
-int forEachEntryInDatabase_Directory(void (*callback)(HashCode160*));
+int forEachEntryInDatabase_DirDB(struct contentDatabase *contentDB,
+                                    void (*callback)(HashCode160*));
  
 #endif
 
-/* end of contentdatabase_gdbm.h */
+/* end of contentdatabase_directory.h */
diff -Naur GNUnet/src/include/util/contentdatabase_gdbm.h 
GNUnet.plugin/src/include/util/contentdatabase_gdbm.h
--- GNUnet/src/include/util/contentdatabase_gdbm.h      Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/include/util/contentdatabase_gdbm.h       Fri May 17 
23:34:28 2002
@@ -18,7 +18,7 @@
 */
 /**
  * ContentDatabse - management of the 1k blocks using a GDBM database.
- * @author Christian Grothoff
+ * @author Uli Luckas, Christian Grothoff
  * @file include/contentdatabase_gdbm.h
  **/
 
@@ -38,7 +38,7 @@
  * @param dir the name of the directory/file
  *        containing the content database
  **/
-void initContentDatabase_GDBM(char * dir);
+struct contentDatabase *initContentDatabase_GDBM(char * dir);
 
 /**
  * Clean shutdown of the storage module (not used at the moment)
@@ -49,7 +49,8 @@
  * Free space in the database by removing one file
  * @param fn the name of the file (without directory)
  **/
-void unlinkFromDB_GDBM(HashCode160 * fn);
+void unlinkFromDB_GDBM(struct contentDatabase *,
+                      HashCode160 * fn);
  
 /**
  * Read the contents of a bucket to a buffer.
@@ -58,7 +59,8 @@
  * @param result the buffer to write the result to
  * @return the number of bytes read on success, -1 on failure
  **/ 
-int readContent_GDBM(HashCode160 * fn,
+int readContent_GDBM(struct contentDatabase *,
+                    HashCode160 * fn,
                     int  len,
                     void * result);
 
@@ -68,7 +70,8 @@
  * @param fn the name of the file (without directory)
  * @param block the CONTENT_SIZE bytes long block (no header!)
  **/
-void writeContent_GDBM(HashCode160 * fn, 
+void writeContent_GDBM(struct contentDatabase *,
+                      HashCode160 * fn, 
                       CONTENT_Block * block);
 
 /**
@@ -76,7 +79,8 @@
  * call the callback method on it. 
  * @return the number of items stored in the content database
  **/
-int forEachEntryInDatabase_GDBM(void (*callback)(HashCode160*));
+int forEachEntryInDatabase_GDBM(struct contentDatabase *,
+                               void (*callback)(HashCode160*));
  
 #endif
 
diff -Naur GNUnet/src/include/util/contentdatabase_tdb.h 
GNUnet.plugin/src/include/util/contentdatabase_tdb.h
--- GNUnet/src/include/util/contentdatabase_tdb.h       Thu Jan  1 01:00:00 1970
+++ GNUnet.plugin/src/include/util/contentdatabase_tdb.h        Fri May 17 
23:34:26 2002
@@ -0,0 +1,84 @@
+/*
+     This file is part of GNUnet
+
+     GNUnet is free software; you can redistribute it and/or modify
+     it under the terms of the GNU General Public License as published
+     by the Free Software Foundation; either version 2, or (at your
+     option) any later version.
+
+     GNUnet is distributed in the hope that it will be useful, but
+     WITHOUT ANY WARRANTY; without even the implied warranty of
+     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+     General Public License for more details.
+
+     You should have received a copy of the GNU General Public License
+     along with GNUnet; see the file COPYING.  If not, write to the
+     Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+     Boston, MA 02111-1307, USA.
+*/
+/**
+ * ContentDatabse - management of the 1k blocks...
+ * @author Uli Luckas, Christian Grothoff
+ * @file gnet/include/storage.h
+ **/
+
+#ifndef CONTENTDATABASE_TDB_H
+#define CONTENTDATABASE_TDB_H
+
+#include "config.h"
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <string.h>
+#include <openssl/ripemd.h>
+#ifndef SOMEBSD
+#include <wordexp.h>
+#endif
+#include <tdb.h>
+
+#include "storage.h"
+
+/**
+ * init the storage module.
+ * @param dir the name of the directory/file
+ *        containing the content database
+ **/
+void initContentDatabase_TDB(DirName * dir);
+
+/**
+ * Free space in the database by removing one file
+ * @param fn the name of the file (without directory)
+ **/
+void unlinkFromDB(HexName *fn);
+ 
+/**
+ * Read the contents of a bucket to a buffer.
+ * @param fileName the name of the file, not freed!
+ * @param len the maximum number of bytes to read
+ * @param result the buffer to write the result to
+ * @return the number of bytes read on success, -1 on failure
+ **/ 
+int readContent(HexName *fn,
+               int  len,
+               void * result);
+
+/**
+ * Write content to a file. Check for reduncancy and eventually
+ * append.
+ * @param fn the name of the file (without directory)
+ * @param block the CONTENT_SIZE bytes long block (no header!)
+ **/
+void writeContent(HexName * fn, 
+                 CONTENT_Block * block);
+
+/**
+ * Call a method for each entry in the database and
+ * call the callback method on it. 
+ * @return the number of items stored in the content database
+ **/
+int forEachEntryInDatabase(void (*callback)(HashCode160*));
+ 
+#endif
+
+/* end of contentdatabase_tdb.h */
diff -Naur GNUnet/src/server/gnet.c GNUnet.plugin/src/server/gnet.c
--- GNUnet/src/server/gnet.c    Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/server/gnet.c     Fri May 17 19:11:34 2002
@@ -133,8 +133,7 @@
   readConfig(configFile);
 
   checkCompiler();
-  initContentDatabase((char*)expandFileName
-                     ((FileName)getContentDirectory()));
+  initContentDatabase(getDatabaseType(), getContentDirectory());
   initCron();
   initTCPIO();
   initStatusCalls(getNetInterfaces(),
diff -Naur GNUnet/src/textui/insertmain.c GNUnet.plugin/src/textui/insertmain.c
--- GNUnet/src/textui/insertmain.c      Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/textui/insertmain.c       Fri May 17 19:17:11 2002
@@ -122,7 +122,7 @@
   description = argv[optind++];
   keywords = &(argv[optind]);
   num_keywords = argc - optind;
-  initContentDatabase(getContentDirectory());
+  initContentDatabase(getDatabaseType(), getContentDirectory());
   initCron();
   initTCPIO();
   initLookup();
diff -Naur GNUnet/src/textui/insertmp3.c GNUnet.plugin/src/textui/insertmp3.c
--- GNUnet/src/textui/insertmp3.c       Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/textui/insertmp3.c        Fri May 17 19:17:45 2002
@@ -225,7 +225,7 @@
   }
   readConfig(confFile);
   initRAND(); 
-  initContentDatabase(getContentDirectory());
+  initContentDatabase(getDatabaseType(), getContentDirectory());
   initCron();
   initTCPIO();
   initLookup();  
diff -Naur GNUnet/src/util/contentdatabase.c 
GNUnet.plugin/src/util/contentdatabase.c
--- GNUnet/src/util/contentdatabase.c   Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/util/contentdatabase.c    Sat May 18 00:06:43 2002
@@ -19,7 +19,7 @@
 */
 
 /**
- * ContentDatabase (dispatch implementation, currently bound to GDBM). 
+ * ContentDatabase (dispatch implementation). 
  * @author Christian Grothoff
  * @file util/contentdatabase.c
  **/
@@ -27,6 +27,7 @@
 #include "config.h"
 #include "contentdatabase.h"
 #include "contentdatabase_gdbm.h"
+#include "contentdatabase_tdb.h"
 #include "xmalloc.h"
 #include "semaphore.h"
 
@@ -43,16 +44,29 @@
 #include <dirent.h>
 #include <errno.h>
 
+struct contentDatabase *contentDB = NULL;
+
+inline void initContentDatabase(char * dbType, char * dir) {
+  if (strcmp(dbType, "dir") == 0) {
+    contentDB = initContentDatabase_DirDB(dir);
+  } else if (strcmp(dbType, "gdbm") == 0) {
+    contentDB = initContentDatabase_GDBM(dir);
+    //  } else if (strcmp(dbType, "tdb") == 0) {
+    //    contentDB = initContentDatabase_TDB(dir);
+  } else {
+      fprintf(stderr,
+         "No handler for '%s' content databases found.\n", 
+         dbType);
+      exit(-1);
+  }
 
-void initContentDatabase(char * dir) {  
-  initContentDatabase_GDBM(dir);
 }
 
 /**
  * Clean shutdown of the storage module (not used at the moment)
  **/
-void doneContentDatabase() {
-  doneContentDatabase_GDBM();
+inline void doneContentDatabase() {
+  contentDB->doneContentDatabase(contentDB);
 }
 
 /**
@@ -60,8 +74,8 @@
  * call the callback method on it. 
  * @return the number of items stored in the content database
  **/
-int forEachEntryInDatabase(void (*callback)(HashCode160*)) {
-  return forEachEntryInDatabase_GDBM(callback);
+inline int forEachEntryInDatabase(void (*callback)(HashCode160*)) {
+  return contentDB->forEachEntryInDatabase(contentDB,callback);
 }
 
 /**
@@ -71,10 +85,10 @@
  * @param result the buffer to write the result to
  * @return the number of bytes read on success, -1 on failure
  **/ 
-int readContent(HashCode160 * name,
+inline int readContent(HashCode160 * name,
                int  len,
                void * result) {
-  return readContent_GDBM(name,len,result);
+  return contentDB->readContent(contentDB,name,len,result);
 }
 
 /**
@@ -83,18 +97,17 @@
  * @param fn the name of the file (without directory)
  * @param block the CONTENT_SIZE bytes long block (no header!)
  **/
-void writeContent(HashCode160 * name, 
+inline void writeContent(HashCode160 * name, 
                  CONTENT_Block * block) {
-  writeContent_GDBM(name, block);
+  contentDB->writeContent(contentDB, name, block);
 }
 
 /**
  * Free space in the database by removing one file
  * @param fn the name of the file (without directory)
  **/
-void unlinkFromDB(HashCode160 * name) {
-  unlinkFromDB_GDBM(name);
+inline void unlinkFromDB(HashCode160 * name) {
+  contentDB->unlinkFromDB(contentDB, name);
 }
-
 
 /* end of contentdatabase.c */
diff -Naur GNUnet/src/util/contentdatabase_directory.c 
GNUnet.plugin/src/util/contentdatabase_directory.c
--- GNUnet/src/util/contentdatabase_directory.c Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/util/contentdatabase_directory.c  Fri May 17 23:26:17 2002
@@ -21,10 +21,11 @@
 /**
  * ContentDatabase (directory based implementation). 
  * @author Christian Grothoff
- * @file util/contentdatabase_gdbm.c
+ * @file util/contentdatabase_directory.c
  **/
 
 #include "config.h"
+#include "contentdatabase.h"
 #include "contentdatabase_directory.h"
 #include "xmalloc.h"
 #include "semaphore.h"
@@ -45,7 +46,11 @@
 /**
  * The (expanded) directory name (i.e. "/home/usr/.gnunet/data/content/")
  **/
-static char * directory_name;
+struct dirDB_private_data {
+  char * directory_name;
+};
+
+#define dirDBData(contentDB) ((struct dirDB_private_data *) 
contentDB->privateData)
 
 /**
  * Initialize the Directory module, expand filename
@@ -62,28 +67,46 @@
                "getDirectory: filename (tmp)");  
   tmp[0] = '\0';
   strcat(tmp, dir);
-  strcat(tmp,"\\");
+  strcat(tmp,"/");
   result = expandFileName(tmp);
   xfree(tmp, "getDirectory: filename (tmp)");
   return result;
 }
 
-void initContentDatabase_Directory(char * dir) {  
-  directory_name = getDirectory(dir);
-  if (directory_name == NULL) {
+struct contentDatabase *initContentDatabase_DirDB(char * dir) {  
+
+  struct contentDatabase *contentDB;
+  contentDB = xmalloc(sizeof(*contentDB),
+                     "initContentDatabase_DirDB: contentDB");
+  contentDB->doneContentDatabase = doneContentDatabase_DirDB;
+  contentDB->unlinkFromDB = unlinkFromDB_DirDB;
+  contentDB->readContent = readContent_DirDB;
+  contentDB->writeContent = writeContent_DirDB;
+  contentDB->forEachEntryInDatabase = forEachEntryInDatabase_DirDB;
+  dirDBData(contentDB) = xmalloc(sizeof(*dirDBData(contentDB)),
+                                  "initContentDatabase_DirDB: dirDB_data");
+  dirDBData(contentDB)->directory_name = getDirectory(dir);
+  if (dirDBData(contentDB)->directory_name == NULL) {
+    xfree(dirDBData(contentDB),
+         "initContentDatabase_DirDB: dirDB_data");
+    xfree(contentDB,
+         "initContentDatabase_DirDB: contentDB");
     fprintf(stderr,
            "FATAL: could not open directory %s!\n",
            (char*) dir);
     exit(-1);
   }
+  return contentDB;
 }
 
 /**
  * Clean shutdown of the storage module (not used at the moment)
  **/
-void doneContentDatabase_Directory() {
-  xfree(directory_name,
-       "doneContentDatabase_Directory");
+void doneContentDatabase_DirDB(struct contentDatabase *contentDB) {
+  xfree(dirDBData(contentDB),
+         "initContentDatabase_DirDB: dirDB_data");
+  xfree(contentDB,
+         "initContentDatabase_DirDB: contentDB");
 }
 
 /**
@@ -91,14 +114,15 @@
  * call the callback method on it. 
  * @return the number of items stored in the content database
  **/
-int forEachEntryInDatabase_Directory(void (*callback)(HashCode160*)) {
+int forEachEntryInDatabase_DirDB(struct contentDatabase *contentDB,
+                                    void (*callback)(HashCode160*)) {
   DIR * dinfo;
   struct dirent *finfo;
   struct stat istat;
   int count;
   HashCode160 hash;
 
-  stat(directory_name,&istat);
+  stat(dirDBData(contentDB)->directory_name,&istat);
   if (!S_ISDIR(istat.st_mode)) {
 #ifdef PRINT_WARNINGS
     fprintf(stderr,
@@ -108,11 +132,11 @@
     return -1;
   }
   errno = 0;
-  dinfo = opendir(directory_name);
+  dinfo = opendir(dirDBData(contentDB)->directory_name);
   if ((errno == EACCES) || (dinfo == NULL)) {
     fprintf(stderr,
             "scanDirectory: access denied (%s)\n",
-            directory_name);
+            dirDBData(contentDB)->directory_name);
     return -1;
   }
   count = 0;
@@ -135,7 +159,8 @@
  * @param result the buffer to write the result to
  * @return the number of bytes read on success, -1 on failure
  **/ 
-int readContent_Directory(HashCode160 * name,
+int readContent_DirDB(struct contentDatabase *contentDB,
+                         HashCode160 * name,
                          int  len,
                          void * result) {
   /* open file, must exist, open read only */
@@ -147,9 +172,9 @@
   if ((name == NULL) || (result == NULL))
     return -1;
   hash2hex(name, &fn);
-  fil = xmalloc(strlen(directory_name) + strlen((char*)&fn) + 1,
-               "readContent_Directory: filename");
-  buildFileName(directory_name, &fn, fil);
+  fil = xmalloc(strlen(dirDBData(contentDB)->directory_name) + 
strlen((char*)&fn) + 1,
+               "readContent_DirDB: filename");
+  buildFileName(dirDBData(contentDB)->directory_name, &fn, fil);
   handle = open(fil,O_RDONLY,S_IRUSR);
   xfree(fil,"readContent: filename");
 
@@ -166,7 +191,8 @@
  * @param fn the name of the file (without directory)
  * @param block the CONTENT_SIZE bytes long block (no header!)
  **/
-void writeContent_Directory(HashCode160 * name, 
+void writeContent_DirDB(struct contentDatabase *contentDB,
+                           HashCode160 * name, 
                            CONTENT_Block * block) {
   CONTENT_Block preBlock[MAX_FSIZE];
   HexName fn;
@@ -177,9 +203,9 @@
   int handle;
 
   hash2hex(name, &fn);
-  fil = xmalloc(strlen((char*)directory_name) + strlen((char*)&fn) + 1,
-               "writeContent_Directory: filename");
-  buildFileName(directory_name, &fn, fil);
+  fil = xmalloc(strlen((char*)dirDBData(contentDB)->directory_name) + 
strlen((char*)&fn) + 1,
+               "writeContent_DirDB: filename");
+  buildFileName(dirDBData(contentDB)->directory_name, &fn, fil);
   handle = open(fil, O_RDWR|O_CREAT, S_IRUSR|S_IWUSR);
   if (handle == -1) {
     fprintf(stderr,
@@ -187,7 +213,7 @@
     xfree(fil,"writeContent: filaname (1)");
     return; /* failed! */
   }
-  xfree(fil,"writeContent_Directory: filename");
+  xfree(fil,"writeContent_DirDB: filename");
   j = read(handle, preBlock, MAX_FSIZE*CONTENT_SIZE);
   if ((j == 0)||(j % CONTENT_SIZE != 0)) {
     write(handle, block, CONTENT_SIZE);
@@ -219,19 +245,23 @@
  * Free space in the database by removing one file
  * @param fn the name of the file (without directory)
  **/
-void unlinkFromDB_Directory(HashCode160 * name) {
+void unlinkFromDB_DirDB(struct contentDatabase *contentDB,
+                           HashCode160 * name) {
   HexName fn;
   FileName fil;
 
   if (name == NULL)
     return;
   hash2hex(name, &fn);
-  fil = xmalloc(strlen(directory_name) + strlen((char*)&fn) + 1,
-               "unlinkFromDB_Directory: filename");
-  buildFileName(directory_name, &fn, fil);
+  fil = xmalloc(strlen(dirDBData(contentDB)->directory_name) + 
strlen((char*)&fn) + 1,
+               "unlinkFromDB_DirDB: filename");
+  buildFileName(dirDBData(contentDB)->directory_name, &fn, fil);
   unlink(fil);
-  xfree(fil,"unlinkFromDB_Directory: filename");
+  xfree(fil,"unlinkFromDB_DirDB: filename");
 }
 
 
 /* end of contentdatabase_gdbm.c */
+
+
+
diff -Naur GNUnet/src/util/contentdatabase_gdbm.c 
GNUnet.plugin/src/util/contentdatabase_gdbm.c
--- GNUnet/src/util/contentdatabase_gdbm.c      Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/util/contentdatabase_gdbm.c       Fri May 17 23:35:06 2002
@@ -20,12 +20,12 @@
 
 /**
  * ContentDatabase (gdbm implementation). 
- * @author Christian Grothoff
- * @author Uli Luckas
+ * @author Uli Luckas, Christian Grothoff
  * @file util/contentdatabase_gdbm.c
  **/
 
 #include "config.h"
+#include "contentdatabase.h"
 #include "contentdatabase_gdbm.h"
 #include "xmalloc.h"
 #include "semaphore.h"
@@ -43,13 +43,16 @@
 #include <dirent.h>
 #include <errno.h>
 
-static GDBM_FILE dbf;
+struct gdbm_private_data {
+  GDBM_FILE dbf;
 
 /**
  * Mutex for locking on requestmanager shared state
  **/
-static Mutex DATABASE_Lock_;
+  Mutex DATABASE_Lock_;
+};
 
+#define gdbmData(contentDB) ((struct gdbm_private_data *) 
contentDB->privateData)
 
 /**
  * Open a gdbm database (for content)
@@ -97,22 +100,42 @@
   return result;
 }
 
-void initContentDatabase_GDBM(char * dir) {  
-  dbf = getDatabase(dir);
-  if (dbf == NULL) {
+struct contentDatabase *initContentDatabase_GDBM(char * dir) {  
+
+  struct contentDatabase *contentDB;
+  contentDB = xmalloc(sizeof(*contentDB),
+                     "initContentDatabase_GDBM: contentDB");
+  contentDB->doneContentDatabase = doneContentDatabase_GDBM;
+  contentDB->unlinkFromDB = unlinkFromDB_GDBM;
+  contentDB->readContent = readContent_GDBM;
+  contentDB->writeContent = writeContent_GDBM;
+  contentDB->forEachEntryInDatabase = forEachEntryInDatabase_GDBM;
+  gdbmData(contentDB) = xmalloc(sizeof(*gdbmData(contentDB)),
+                                  "initContentDatabase_GDBM: gdbm_data");
+  gdbmData(contentDB)->dbf = getDatabase(dir);
+  if (gdbmData(contentDB)->dbf == NULL) {
+    xfree(gdbmData(contentDB),
+         "initContentDatabase_GDBM: gdbm_data");
+    xfree(contentDB,
+         "initContentDatabase_GDBM: contentDB");
     fprintf(stderr,
            "FATAL: could not open database!\n");
     exit(-1);
   }
-  create_recursive_mutex(&DATABASE_Lock_);
+  create_recursive_mutex(&(gdbmData(contentDB)->DATABASE_Lock_));
+  return contentDB;
 }
 
 /**
  * Clean shutdown of the storage module (not used at the moment)
  **/
-void doneContentDatabase_GDBM() {
-  gdbm_close(dbf);
-  destroy_mutex(&DATABASE_Lock_);
+void doneContentDatabase_GDBM(struct contentDatabase *contentDB) {
+  gdbm_close(gdbmData(contentDB)->dbf);
+  destroy_mutex(&(gdbmData(contentDB)->DATABASE_Lock_));
+  xfree(gdbmData(contentDB),
+         "initContentDatabase_GDBM: gdbm_data");
+  xfree(contentDB,
+         "initContentDatabase_GDBM: contentDB");
 }
 
 /**
@@ -120,7 +143,7 @@
  * call the callback method on it. 
  * @return the number of items stored in the content database
  **/
-int forEachEntryInDatabase_GDBM(void (*callback)(HashCode160*)) {
+int forEachEntryInDatabase_GDBM(struct contentDatabase *contentDB, void 
(*callback)(HashCode160*)) {
   datum prevkey, key;
   HashCode160 doubleHash;
   int count;
@@ -128,8 +151,8 @@
   count = 0;
   /* scan database data/content.dbf and add entries to database 
      if not already present */
-  mutex_lock(&DATABASE_Lock_);
-  key = gdbm_firstkey(dbf);
+  mutex_lock(&(gdbmData(contentDB)->DATABASE_Lock_));
+  key = gdbm_firstkey(gdbmData(contentDB)->dbf);
   while (key.dptr) {
     if (strlen(key.dptr) == sizeof(HashCode160)*2) {   
       hex2hash((HexName*)key.dptr,
@@ -138,11 +161,11 @@
       count++; /* one more file */
     }
     prevkey = key;
-    key = gdbm_nextkey(dbf, prevkey);
+    key = gdbm_nextkey(gdbmData(contentDB)->dbf, prevkey);
     xfree(prevkey.dptr, "database dptr (allocation in DB code)");
   }
-  mutex_unlock(&DATABASE_Lock_);
-  key = gdbm_firstkey(dbf);
+  mutex_unlock(&(gdbmData(contentDB)->DATABASE_Lock_));
+  key = gdbm_firstkey(gdbmData(contentDB)->dbf);
   return count;
 }
 
@@ -153,7 +176,8 @@
  * @param result the buffer to write the result to
  * @return the number of bytes read on success, -1 on failure
  **/ 
-int readContent_GDBM(HashCode160 * name,
+int readContent_GDBM(struct contentDatabase *contentDB,
+                    HashCode160 * name,
                     int  len,
                     void * result) {
   HexName fn;
@@ -167,9 +191,9 @@
   key.dptr = fn.data;
   key.dsize = strlen(key.dptr) + 1;
 
-  mutex_lock(&DATABASE_Lock_);
-  buffer = gdbm_fetch(dbf, key);
-  mutex_unlock(&DATABASE_Lock_);
+  mutex_lock(&(gdbmData(contentDB)->DATABASE_Lock_));
+  buffer = gdbm_fetch(gdbmData(contentDB)->dbf, key);
+  mutex_unlock(&(gdbmData(contentDB)->DATABASE_Lock_));
   if (!buffer.dptr) 
     return -1;
   if (len > buffer.dsize) 
@@ -185,7 +209,8 @@
  * @param fn the name of the file (without directory)
  * @param block the CONTENT_SIZE bytes long block (no header!)
  **/
-void writeContent_GDBM(HashCode160 * name, 
+void writeContent_GDBM(struct contentDatabase *contentDB,
+                      HashCode160 * name, 
                       CONTENT_Block * block) {
   CONTENT_Block preBlock[MAX_FSIZE];
   HexName fn;
@@ -197,9 +222,9 @@
   hash2hex(name, &fn);
   key.dptr = fn.data;
   key.dsize = strlen(key.dptr) + 1;
-  mutex_lock(&DATABASE_Lock_);
-  buffer = gdbm_fetch(dbf, key);
-  mutex_unlock(&DATABASE_Lock_);
+  mutex_lock(&(gdbmData(contentDB)->DATABASE_Lock_));
+  buffer = gdbm_fetch(gdbmData(contentDB)->dbf, key);
+  mutex_unlock(&(gdbmData(contentDB)->DATABASE_Lock_));
   if (!buffer.dptr) {
     j = 0;
   } else {
@@ -221,9 +246,9 @@
     }
     buffer.dptr = block->content;
     buffer.dsize = CONTENT_SIZE;
-    mutex_lock(&DATABASE_Lock_);
-    gdbm_store(dbf, key, buffer, GDBM_REPLACE);
-    mutex_unlock(&DATABASE_Lock_);
+    mutex_lock(&(gdbmData(contentDB)->DATABASE_Lock_));
+    gdbm_store(gdbmData(contentDB)->dbf, key, buffer, GDBM_REPLACE);
+    mutex_unlock(&(gdbmData(contentDB)->DATABASE_Lock_));
     return;
   }
   blocks = j / CONTENT_SIZE;
@@ -254,26 +279,31 @@
   memcpy(preBlock[blocks].content , block->content, CONTENT_SIZE);
   buffer.dptr = preBlock->content;
   buffer.dsize += CONTENT_SIZE;
-  mutex_lock(&DATABASE_Lock_);
-  gdbm_store(dbf, key, buffer, GDBM_REPLACE);
-  mutex_unlock(&DATABASE_Lock_);
+  mutex_lock(&(gdbmData(contentDB)->DATABASE_Lock_));
+  gdbm_store(gdbmData(contentDB)->dbf, key, buffer, GDBM_REPLACE);
+  mutex_unlock(&(gdbmData(contentDB)->DATABASE_Lock_));
 }
 
 /**
  * Free space in the database by removing one file
  * @param fn the name of the file (without directory)
  **/
-void unlinkFromDB_GDBM(HashCode160 * name) {
+void unlinkFromDB_GDBM(struct contentDatabase *contentDB,
+                      HashCode160 * name) {
   datum key;
   HexName fn;
 
   hash2hex(name, &fn);
   key.dptr = fn.data;
   key.dsize = strlen(key.dptr) + 1;
-  mutex_lock(&DATABASE_Lock_);
-  gdbm_delete(dbf, key);
-  mutex_unlock(&DATABASE_Lock_);
+  mutex_lock(&(gdbmData(contentDB)->DATABASE_Lock_));
+  gdbm_delete(gdbmData(contentDB)->dbf, key);
+  mutex_unlock(&(gdbmData(contentDB)->DATABASE_Lock_));
 }
 
 
 /* end of contentdatabase_gdbm.c */
+
+
+
+
diff -Naur GNUnet/src/util/contentdatabase_tdb.c 
GNUnet.plugin/src/util/contentdatabase_tdb.c
--- GNUnet/src/util/contentdatabase_tdb.c       Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/util/contentdatabase_tdb.c        Fri May 17 23:32:35 2002
@@ -20,8 +20,7 @@
 
 /**
  * ContentDatabase (tdb implementation). 
- * @author Christian Grothoff
- * @author Uli Luckas
+ * @author Uli Luckas, Christian Grothoff
  * @file util/contentdatabase_tdb.c
  **/
 
@@ -42,8 +41,11 @@
 #include <dirent.h>
 #include <errno.h>
 
-static TDB_CONTEXT *dbf;
+struct tdb_private_data {
+  TDB_CONTEXT dbf;
+};
 
+#define tdbData(contentDB) ((struct gdbm_private_data *) 
contentDB->privateData)
 
 /**
  * Open a tdb database (for content)
@@ -87,20 +89,40 @@
   return(dbf);
 }
 
-void initContentDatabase_TDB(DirName * dir) {  
-  dbf = getDatabase(dir);
-  if (dbf == NULL) {
+struct contentDatabase *initContentDatabase_TDB(char * dir) {  
+
+  struct contentDatabase *contentDB;
+  contentDB = xmalloc(sizeof(*contentDB),
+                     "initContentDatabase_TDB: contentDB");
+  contentDB->doneContentDatabase = doneContentDatabase_TDB;
+  contentDB->unlinkFromDB = unlinkFromDB_TDB;
+  contentDB->readContent = readContent_TDB;
+  contentDB->writeContent = writeContent_TDB;
+  contentDB->forEachEntryInDatabase = forEachEntryInDatabase_TDB;
+  tdbData(contentDB) = xmalloc(sizeof(*tdbData(contentDB)),
+                                  "initContentDatabase_TDB: tdb_data");
+  tdbData(contentDB)->dbf = getDatabase(dir);
+  if (tdbData(contentDB)->dbf == NULL) {
+    xfree(tdbData(contentDB),
+         "initContentDatabase_TDB: tdb_data");
+    xfree(contentDB,
+         "initContentDatabase_TDB: contentDB");
     fprintf(stderr,
            "FATAL: could not open database!\n");
     exit(-1);
   }
+  return contentDB;
 }
 
 /**
  * Clean shutdown of the storage module (not used at the moment)
  **/
-void doneContentDatabase_TDB() {
-  tdb_close(dbf);
+void doneContentDatabase_TDB(struct contentDatabase *contentDB) {
+  tdb_close(tdbData(contentDB)->dbf);
+  xfree(tdbData(contentDB),
+         "initContentDatabase_TDB: tdb_data");
+  xfree(contentDB,
+         "initContentDatabase_TDB: contentDB");
 }
 
 /**
@@ -108,15 +130,16 @@
  * call the callback method on it. 
  * @return the number of items stored in the content database
  **/
-int forEachEntryInDatabase_TDB(void (*callback)(HashCode160*)) {
+int forEachEntryInDatabase_TDB(struct contentDatabase *contentDB,
+                              void (*callback)(HashCode160*)) {
   TDB_DATA prevkey, key;
   HashCode160 doubleHash;
   int count;
 
   count = 0;
-  /* scan database data/content.dbf and add entries to database 
+  /* scan database data/content.tdbData(contentDB)->dbf and add entries to 
database 
      if not already present */
-  key = tdb_firstkey(dbf);
+  key = tdb_firstkey(tdbData(contentDB)->dbf);
   while (key.dptr) {
     if (strlen(key.dptr) == sizeof(HashCode160)*2) {   
       hex2hash((HexName*)key.dptr,
@@ -125,7 +148,7 @@
       count++; /* one more file */
     }
     prevkey = key;
-    key = tdb_nextkey(dbf, prevkey);
+    key = tdb_nextkey(tdbData(contentDB)->dbf, prevkey);
     xfree(prevkey.dptr, "database dptr (allocation in DB code)");
   }
   return count;
@@ -138,7 +161,8 @@
  * @param result the buffer to write the result to
  * @return the number of bytes read on success, -1 on failure
  **/ 
-int readContent_TDB(HashCode160 * name,
+int readContent_TDB(struct contentDatabase *contentDB,
+                   HashCode160 * name,
                    int  len,
                    void * result) {
   HexName fn;
@@ -152,7 +176,7 @@
   key.dptr = fn->data;
   key.dsize = strlen(key.dptr) + 1;
 
-  buffer = tdb_fetch(dbf, key);
+  buffer = tdb_fetch(tdbData(contentDB)->dbf, key);
   if (!buffer.dptr) 
     return -1;
   if (len > buffer.dsize) 
@@ -168,7 +192,8 @@
  * @param name the name of the file (without directory)
  * @param block the CONTENT_SIZE bytes long block (no header!)
  **/
-void writeContent_TDB(HashCode160 * name, 
+void writeContent_TDB(struct contentDatabase *contentDB,
+                     HashCode160 * name, 
                      CONTENT_Block * block) {
   HexName fn;
   CONTENT_Block preBlock[MAX_FSIZE];
@@ -180,7 +205,7 @@
   hash2hex(name, &fn);
   key.dptr = fn->data;
   key.dsize = strlen(key.dptr) + 1;
-  buffer = tdb_fetch(dbf, key);
+  buffer = tdb_fetch(tdbData(contentDB)->dbf, key);
   if (!buffer.dptr) {
     j = 0;
   } else {
@@ -200,7 +225,7 @@
     }
     buffer.dptr = block->content;
     buffer.dsize = CONTENT_SIZE;
-    tdb_store(dbf, key, buffer, TDB_REPLACE);
+    tdb_store(tdbData(contentDB)->dbf, key, buffer, TDB_REPLACE);
     return;
   }
   blocks = j / CONTENT_SIZE;
@@ -227,14 +252,15 @@
   memcpy(preBlock[blocks].content , block->content, CONTENT_SIZE);
   buffer.dptr = preBlock->content;
   buffer.dsize += CONTENT_SIZE;
-  tdb_store(dbf, key, buffer, TDB_REPLACE);
+  tdb_store(tdbData(contentDB)->dbf, key, buffer, TDB_REPLACE);
 }
 
 /**
  * Free space in the database by removing one file
  * @param fn the name of the file (without directory)
  **/
-void unlinkFromDB_TDB(HashCode160 * name) {
+void unlinkFromDB_TDB(struct contentDatabase *contentDB,
+                     HashCode160 * name) {
   TDB_DATA key;
   HexName fn;
 
@@ -242,8 +268,15 @@
  
   key.dptr = fn->data;
   key.dsize = strlen(key.dptr) + 1;
-  tdb_delete(dbf, key);
+  tdb_delete(tdbData(contentDB)->dbf, key);
 }
 
 
 /* end of contentdatabase_tdb.c */
+
+
+
+
+
+
+
diff -Naur GNUnet/src/util/contentdatabase_tdb.h 
GNUnet.plugin/src/util/contentdatabase_tdb.h
--- GNUnet/src/util/contentdatabase_tdb.h       Fri May 17 02:42:14 2002
+++ GNUnet.plugin/src/util/contentdatabase_tdb.h        Thu Jan  1 01:00:00 1970
@@ -1,84 +0,0 @@
-/*
-     This file is part of GNUnet
-
-     GNUnet is free software; you can redistribute it and/or modify
-     it under the terms of the GNU General Public License as published
-     by the Free Software Foundation; either version 2, or (at your
-     option) any later version.
-
-     GNUnet is distributed in the hope that it will be useful, but
-     WITHOUT ANY WARRANTY; without even the implied warranty of
-     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-     General Public License for more details.
-
-     You should have received a copy of the GNU General Public License
-     along with GNUnet; see the file COPYING.  If not, write to the
-     Free Software Foundation, Inc., 59 Temple Place - Suite 330,
-     Boston, MA 02111-1307, USA.
-*/
-/**
- * ContentDatabse - management of the 1k blocks...
- * @author Christian Grothoff
- * @file gnet/include/storage.h
- **/
-
-#ifndef CONTENTDATABASE_TDB_H
-#define CONTENTDATABASE_TDB_H
-
-#include "config.h"
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <unistd.h>
-#include <string.h>
-#include <openssl/ripemd.h>
-#ifndef SOMEBSD
-#include <wordexp.h>
-#endif
-#include <tdb.h>
-
-#include "storage.h"
-
-/**
- * init the storage module.
- * @param dir the name of the directory/file
- *        containing the content database
- **/
-void initContentDatabase_TDB(DirName * dir);
-
-/**
- * Free space in the database by removing one file
- * @param fn the name of the file (without directory)
- **/
-void unlinkFromDB(HexName *fn);
- 
-/**
- * Read the contents of a bucket to a buffer.
- * @param fileName the name of the file, not freed!
- * @param len the maximum number of bytes to read
- * @param result the buffer to write the result to
- * @return the number of bytes read on success, -1 on failure
- **/ 
-int readContent(HexName *fn,
-               int  len,
-               void * result);
-
-/**
- * Write content to a file. Check for reduncancy and eventually
- * append.
- * @param fn the name of the file (without directory)
- * @param block the CONTENT_SIZE bytes long block (no header!)
- **/
-void writeContent(HexName * fn, 
-                 CONTENT_Block * block);
-
-/**
- * Call a method for each entry in the database and
- * call the callback method on it. 
- * @return the number of items stored in the content database
- **/
-int forEachEntryInDatabase(void (*callback)(HashCode160*));
- 
-#endif
-
-/* end of contentdatabase_tdb.h */

reply via email to

[Prev in Thread] Current Thread [Next in Thread]