[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[GNUnet-SVN] [gnunet-texinfo] 02/03: developer.texi: @settitle{} fixes u
|
From: |
gnunet |
|
Subject: |
[GNUnet-SVN] [gnunet-texinfo] 02/03: developer.texi: @settitle{} fixes up to line 4463. |
|
Date: |
Fri, 17 Mar 2017 18:40:51 +0100 |
This is an automated email from the git hooks/post-receive script.
ng0 pushed a commit to branch master
in repository gnunet-texinfo.
commit d1348d277c6bae8ff0dbf360e650eeea95656a2e
Author: ng0 <address@hidden>
AuthorDate: Fri Feb 17 16:58:04 2017 +0000
developer.texi: @settitle{} fixes up to line 4463.
---
developer.texi | 1769 +++++++++++++++++++++-----------------------------------
1 file changed, 654 insertions(+), 1115 deletions(-)
diff --git a/developer.texi b/developer.texi
index 2107926..ba5d920 100644
--- a/developer.texi
+++ b/developer.texi
@@ -426,8 +426,7 @@ that this description also lists projects that are far from
complete, including
even those that have literally not a single line of code in them yet.
GNUnet sub-projects in order of likely relevance are currently:
address@hidden
address@hidden
address@hidden @asis
@item svn/gnunet Core of the P2P framework, including file-sharing, VPN and
chat applications; this is what the developer handbook covers mostly
@@ -465,7 +464,8 @@ svn/gauger/ Tool for performance regression analysis
@item svn/monkey/ Tool for
automated debugging of distributed systems
@item svn/libmwmodem/ Library for
-accessing satellite connection quality reports @end table
+accessing satellite connection quality reports
address@hidden table
Finally, there are various external projects (see links for a list of those
@@ -698,26 +698,6 @@ service, isolating their host process from errors by the
service. If the
service process crashes, other services and daemons around it should not also
fail, but instead wait for the service process to be restarted by ARM.
address@hidden @columnfractions 0.333333333333333 0.333333333333333
-0.333333333333333
-
address@hidden Attachment
-
address@hidden Size
-
address@hidden daemon_lego_block.png
-
address@hidden 12.9 KB
-
address@hidden service_lego_block.png
-
address@hidden 19.56 KB
-
address@hidden service_stack.png
-
address@hidden 23.32 KB
-
address@hidden multitable
@c ***************************************************************************
@node Subsystem stability
@@ -1076,7 +1056,8 @@ what the API/protocol looks like today
@item unknown someone should think about
where this subsystem is headed
@item n/a this subsystem does not have an
-API/IPC-protocol/P2P-protocol @end table
+API/IPC-protocol/P2P-protocol
address@hidden table
@c ***************************************************************************
@node Naming conventions and coding style guide
@@ -1173,6 +1154,7 @@ libraries might see them via headers; they must be
either@ declared/defined in
C source files or in headers that are in@ the respective directory under
src/modulename/ and NEVER be@ declared in src/include/.
@end itemize
+
@node testcases
@chapter testcases
@@ -1268,8 +1250,9 @@ only one way to write most branches correctly.
@item Combined assignments and tests are allowed if they do not hinder code
clarity. For example, one can write:@
address@hidden if (NULL == (value = lookup_function())) @{ error(); return; @}
@end
-example
address@hidden
+if (NULL == (value = lookup_function())) @{ error(); return; @}
address@hidden example
@item Use @code{break} and @code{continue} wherever possible to avoid deep(er)
@@ -1384,11 +1367,6 @@ If you want to obtain code coverage results, run
configure with the@
@node Developing extensions for GNUnet using the gnunet-ext template
@chapter Developing extensions for GNUnet using the gnunet-ext template
address@hidden @bullet
address@hidden Matthias Wachs's blog
address@hidden
address@hidden itemize
-
For developers who want to write extensions for GNUnet we provide the
gnunet-ext template to provide an easy to use skeleton.
@@ -1681,9 +1659,7 @@ The TESTBED subsystem facilitates testing and measuring
of multi-peer
deployments on a single host or over multiple hosts.
The architecture of the testbed module is divided into the following:
address@hidden
address@hidden
-
address@hidden @bullet
@item Testbed API: An API which is used by the testing driver programs. It
provides with functions for creating, destroying, starting, stopping peers,
@@ -1719,21 +1695,15 @@ GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes -o
NoHostAuthenticationForLocalhost=yes %h"@ }@ Substitutions are allowed int the
above command string also allows for substitions. through placemarks which
begin with a `%'. At present the following substitutions are supported
address@hidden
address@hidden
-
-
address@hidden @bullet
@item
%h: hostname
-
@item
%u: username
-
@item
%p: port
@end itemize
-
Note that the substitution placemark is replaced only when the corresponding
field is available and only once. Specifying @code{%u@@%h} doesn't work either.
If you want to user username substitutions for SSH use the argument @code{-l}
@@ -1778,9 +1748,7 @@ the testbed.
The API also provides a helper function
@code{GNUNET_TESTBED_overlay_configure_topology()} to connect a given set of
peers in any of the following supported topologies:
address@hidden
address@hidden
-
address@hidden @bullet
@item @code{GNUNET_TESTBED_TOPOLOGY_CLIQUE}: All peers are connected with each
other
@@ -1815,7 +1783,8 @@ Networks. Science 286, 509-512, 1999.
loaded from a file. The path to the file has to be given. See Topology file
format for the format of this file.
address@hidden @code{GNUNET_TESTBED_TOPOLOGY_NONE}: No topology @end itemize
address@hidden @code{GNUNET_TESTBED_TOPOLOGY_NONE}: No topology
address@hidden itemize
The above supported topologies can be specified respectively by setting the
@@ -1865,8 +1834,7 @@ hosts file name as its parameter. It uses the above
function to populate the
hosts data structures and start controllers to deploy peers.
These functions require the hosts file to be of the following format:
address@hidden
address@hidden
address@hidden @bullet
@item Each line is interpreted to have details about a host
@item Host details should include the username to use for logging into the
host, the hostname of the host and the port number to use for the remote shell
@@ -1910,8 +1878,7 @@ the barrier. A barrier which is reached is deemed as
crossed after all the
peers waiting on it are notified.
The barriers API provides the following functions:
address@hidden
address@hidden
address@hidden @bullet
@item @address@hidden()}:} function to initialse a
barrier in the experiment
@item @address@hidden()}:} function to cancel a
@@ -2061,8 +2028,7 @@ install. Buildslave will work anyway since version 3.8.0
was installed before!
@chapter Setup a new PlanetLab testbed using GPLMT
@node Setup a new PlanetLab testbed using GPLMT
address@hidden
address@hidden
address@hidden @bullet
@item Get a new slice and assign nodes
Ask your PlanetLab PI to give you a new slice and assign the nodes you need
@item Install a buildmaster
@@ -2176,8 +2142,7 @@ binaries require. The code of libgnunetutil is in the
src/util/ directory. The
public interface to the library is in the gnunet_util.h header. The functions
provided by libgnunetutil fall roughly into the following categories (in
roughly the order of importance for new developers):
address@hidden
address@hidden
address@hidden @bullet
@item logging (common_logging.c)
@item memory allocation (common_allocation.c)
@item endianess conversion (common_endian.c)
@@ -2278,9 +2243,7 @@ suite).@ To fix that, and to allow GNUnet to use
different log filtering at
runtime without re-compiling the whole source tree, the log calls were changed
to be configurable at run time. To configure them one has to define environment
variables "GNUNET_FORCE_LOGFILE", "GNUNET_LOG" and/or "GNUNET_FORCE_LOG":
address@hidden
address@hidden
-
address@hidden @bullet
@item "GNUNET_LOG" only affects the logging when no global log level is
configured by any other means (that is, the process does not explicitly set its
@@ -2295,7 +2258,8 @@ options given.
to log messages to. It should contain a relative or absolute file name. Setting
GNUNET_FORCE_LOGFILE is equivalent to passing "--log-file=logfile" or "-l
logfile" option (see below). It supports "[]" format in file names, but not
-"@address@hidden" (see below). @end itemize
+"@address@hidden" (see below).
address@hidden itemize
Because environment variables are inherited by child processes when they are
@@ -2451,16 +2415,11 @@ never be automatically deleted by GNUnet.
@node Updated behavior of GNUNET_log
@chapter Updated behavior of GNUNET_log
address@hidden
address@hidden
address@hidden Bart Polot's blog
address@hidden itemize
-
-
It's currently quite common to see constructions like this all over the code:
address@hidden MESH_DEBUG GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "MESH: client
-disconnected\n"); #endif @end example
-
address@hidden
+#if MESH_DEBUG GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "MESH: client
+disconnected\n"); #endif
address@hidden example
The reason for the #if is not to avoid displaying the message when disabled
(GNUNET_ERROR_TYPE takes care of that), but to avoid the compiler including it
@@ -2493,8 +2452,7 @@ Pretty hard to follow, huh?
From now on, it is not necessary to include the #if / #endif statements to
acheive the same behavior. The GNUNET_log and GNUNET_log_from macros take care
of it for you, depending on the configure option:
address@hidden
address@hidden
address@hidden @bullet
@item If @code{--enable-logging} is set to @code{no}, the binary will contain
no log messages at all.
@item If @code{--enable-logging} is set to @code{yes}, the binary will contain
@@ -2507,8 +2465,7 @@ neccessary to run with -L DEBUG or set the DEBUG config
option to show them).
If you are a developer:
address@hidden
address@hidden
address@hidden @bullet
@item please make sure that you @code{./configure
address@hidden,address@hidden, so you can see DEBUG messages.
@item please remove the @code{#if} statements around @code{GNUNET_log
@@ -2570,33 +2527,20 @@ Please note @code{GNUNET_NETWORK_STRUCT_BEGIN} and
@code{GNUNET_PACKED} which
both ensure correct alignment when sending structs over the network
@c ***************************************************************************
address@hidden Connection between client and server
address@hidden Connection between client and server
address@hidden Connection between client and server
@c %**end of header
address@hidden Top
-
-
-
For typical communication, the connection should be created first, in other
words, a connection between the client and the service should be
established.
@c ***************************************************************************
address@hidden Client setting
address@hidden Client setting
address@hidden Client setting
@c %**end of header
address@hidden Top
-
@c ***************************************************************************
address@hidden Establish connection
address@hidden Establish connection
address@hidden Establish connection
@c %**end of header
address@hidden Top
-
-
-
At first, on the client side, the underlying API is employed to create a new
connection to a service, in our example the transport service would be
connected.
@@ -2604,19 +2548,17 @@ connected.
struct GNUNET_CLIENT_Connection *client; client =
GNUNET_CLIENT_connect ("transport", cfg);
@end example
+
@c ***************************************************************************
address@hidden Initialize request message
address@hidden Initialize request message
address@hidden Initialize request message
@c %**end of header
address@hidden Top
-
-
-
When the connection is ready, we initialize the message. In this step, all the
fields of the message should be properly initialized, namely the size, type,
and some extra user-defined data, such as timeout, name of transport, address
-and name of transport. @example struct AddressLookupMessage *msg; size_t len =
+and name of transport.
address@hidden
+struct AddressLookupMessage *msg; size_t len =
sizeof (struct AddressLookupMessage) + addressLen + strlen (nameTrans) + 1;
msg->header->size = htons (len); msg->header->type = htons
(GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_LOOKUP); msg->timeout =
@@ -2625,23 +2567,16 @@ char *addrbuf = (char *) &msg[1]; memcpy (addrbuf,
address, addressLen); char
*tbuf = &addrbuf[addressLen]; memcpy (tbuf, nameTrans, strlen (nameTrans) + 1);
@end example
-
-
-
Note that, here the functions @code{htonl}, @code{htons} and
@code{GNUNET_TIME_absolute_hton} are applied to convert little endian into big
endian, about the usage of the big/small edian order and the corresponding
conversion function please refer to Introduction of Big Endian and Little
Endian.
+
@c ***************************************************************************
address@hidden Send request and receive response
address@hidden Send request and receive response
address@hidden Send request and receive response
@c %**end of header
address@hidden Top
-
-
-
Next, the client would send the constructed message as a request to the service
and wait for the response from the service. To accomplish this goal, there are
a number of API calls that can be used. In this example,
@@ -2653,17 +2588,15 @@ GNUNET_CLIENT_transmit_and_get_response
arp_ctx);
@end example
+the argument @code{address_response_processor} is a function with
address@hidden type, which is used to process the reply
+message from the service.
- the argument @code{address_response_processor} is a function with
- @code{GNUNET_CLIENT_MessageHandler} type, which is used to process the reply
- message from the service.
@c ***************************************************************************
address@hidden Server Setting
address@hidden Server Setting
address@hidden Server Setting
@c ***************************************************************************
address@hidden Startup service
address@hidden Startup service
address@hidden Startup service
After receiving the request message, we run a standard GNUnet service startup
sequence using @code{GNUNET_SERVICE_run}, as follows,
@@ -2674,31 +2607,28 @@ GNUNET_SERVICE_OPTION_NONE, &run, NULL)); @}
@end example
@c ***************************************************************************
address@hidden Add new handles for specified messages
address@hidden Add new handles for specified messages
address@hidden Add new handles for specified messages
@c %**end of header
address@hidden Top
-
-
-
in the function above the argument @code{run} is used to initiate transport
service,and defined like this: @example static void run (void *cls, struct
GNUNET_SERVER_Handle *serv, const struct GNUNET_CONFIGURATION_Handle *cfg) @{
GNUNET_SERVER_add_handlers (serv, handlers); @} @end example
- Here, @code{GNUNET_SERVER_add_handlers} must be called in the run function to
- add new handlers in the service. The parameter @code{handlers} is a list of
- @code{struct GNUNET_SERVER_MessageHandler} to tell the service which function
- should be called when a particular type of message is received, and should be
- defined in this way: @example static struct GNUNET_SERVER_MessageHandler
- handlers[] = @{ @{&handle_start, NULL, GNUNET_MESSAGE_TYPE_TRANSPORT_START,
- address@hidden, @{&handle_send, NULL, GNUNET_MESSAGE_TYPE_TRANSPORT_SEND,
address@hidden,
- @{&handle_try_connect, NULL, GNUNET_MESSAGE_TYPE_TRANSPORT_TRY_CONNECT, sizeof
- (struct TryConnectMessage)@}, @{&handle_address_lookup, NULL,
- GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_LOOKUP, address@hidden, @{NULL, NULL,
0, address@hidden @};
- @end example
+Here, @code{GNUNET_SERVER_add_handlers} must be called in the run function to
+add new handlers in the service. The parameter @code{handlers} is a list of
address@hidden GNUNET_SERVER_MessageHandler} to tell the service which function
+should be called when a particular type of message is received, and should be
+defined in this way:
address@hidden
+static struct GNUNET_SERVER_MessageHandler
+handlers[] = @{ @{&handle_start, NULL, GNUNET_MESSAGE_TYPE_TRANSPORT_START,
address@hidden, @{&handle_send, NULL, GNUNET_MESSAGE_TYPE_TRANSPORT_SEND,
address@hidden,
address@hidden&handle_try_connect, NULL,
GNUNET_MESSAGE_TYPE_TRANSPORT_TRY_CONNECT, sizeof
+(struct TryConnectMessage)@}, @{&handle_address_lookup, NULL,
+GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_LOOKUP, address@hidden, @{NULL, NULL, 0,
address@hidden @};
address@hidden example
As shown, the first member of the struct in the first area is a callback
@@ -2709,16 +2639,11 @@ expected size of the message of this type, usually we
set it to 0 to accept
variable size, for special cases the exact size of the specified message also
can be set. In addition, the terminator sign depicted as @address@hidden,
NULL, 0,
address@hidden is set in the last aera.
+
@c ***************************************************************************
address@hidden Process request message
address@hidden Process request message
address@hidden Process request message
@c %**end of header
-
address@hidden Top
-
-
-
After the initialization of transport service, the request message would be
processed. Before handling the main message data, the validity of this message
should be checked out, e.g., to check whether the size of message is correct.
@@ -2735,13 +2660,15 @@ detail please refer to Introduction of Big Endian and
Little Endian.
Moreover in this example, the name of the transport stored in the message is a
0-terminated string, so we should also check whether the name of the transport
-in the received message is 0-terminated: @examplenameTransport = (const char *)
+in the received message is 0-terminated:
address@hidden
+nameTransport = (const char *)
&address[addressLen]; if (nameTransport[size - sizeof (struct
AddressLookupMessage)
- addressLen - 1] != '\0') @{ GNUNET_break_op
(0); GNUNET_SERVER_receive_done (client,
- GNUNET_SYSERR); return; @} @end example
-
+ GNUNET_SYSERR); return; @}
address@hidden example
Here, @code{GNUNET_SERVER_receive_done} should be called to tell the service
that the request is done and can receive the next message. The argument
@@ -2752,20 +2679,17 @@ In comparison to the aforementioned situation, when the
argument is equal to
@code{GNUNET_OK}, the service would continue to process the requst message.
@c ***************************************************************************
address@hidden Response to client
address@hidden Response to client
address@hidden Response to client
@c %**end of header
address@hidden Top
-
-
-
Once the processing of current request is done, the server should give the
response to the client. A new @code{struct AddressLookupMessage} would be
produced by the server in a similar way as the client did and sent to the
client, but here the type should be
@code{GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY} rather than
address@hidden in client. @example struct
address@hidden in client.
address@hidden
+struct
AddressLookupMessage *msg; size_t len = sizeof (struct AddressLookupMessage) +
addressLen + strlen (nameTrans) + 1; msg->header->size = htons (len);
msg->header->type = htons (GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY);
@@ -2776,28 +2700,25 @@ struct GNUNET_SERVER_TransmitContext *tc; tc =
GNUNET_SERVER_transmit_context_create (client);
GNUNET_SERVER_transmit_context_append_data (tc, NULL, 0,
GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY);
-GNUNET_SERVER_transmit_context_run (tc, rtimeout); @end example
+GNUNET_SERVER_transmit_context_run (tc, rtimeout);
address@hidden example
- Note that, there are also a number of other APIs provided to the service to
- send the message.
+Note that, there are also a number of other APIs provided to the service to
+send the message.
+
@c ***************************************************************************
address@hidden Notification of clients
address@hidden Notification of clients
address@hidden Notification of clients
@c %**end of header
address@hidden Top
-
-
-
- Often a service needs to (repeatedly) transmit notifications to a client or a
- group of clients. In these cases, the client typically has once registered for
- a set of events and then needs to receive a message whenever such an event
- happens (until the client disconnects). The use of a notification context can
- help manage message queues to clients and handle disconnects. Notification
- contexts can be used to send individualized messages to a particular client or
- to broadcast messages to a group of clients. An individualized notification
- might look like this:
+Often a service needs to (repeatedly) transmit notifications to a client or a
+group of clients. In these cases, the client typically has once registered for
+a set of events and then needs to receive a message whenever such an event
+happens (until the client disconnects). The use of a notification context can
+help manage message queues to clients and handle disconnects. Notification
+contexts can be used to send individualized messages to a particular client or
+to broadcast messages to a group of clients. An individualized notification
+might look like this:
@example
GNUNET_SERVER_notification_context_unicast(nc,
client, msg, GNUNET_YES);
@@ -2808,43 +2729,32 @@ Note that after processing the original registration
message for notifications,
the server code still typically needs to call@
@code{GNUNET_SERVER_receive_done} so that the client can transmit further
messages to the server.
address@hidden
address@hidden
-
-
address@hidden
address@hidden itemize
@c ***************************************************************************
address@hidden Conversion between Network Byte Order (Big Endian) and Host Byte
Order
address@hidden Conversion between Network Byte Order (Big Endian) and Host Byte
Order
address@hidden Conversion between Network Byte Order (Big Endian) and Host Byte
Order
@c %**end of header
address@hidden Top
-
-
-
Here we can simply comprehend big endian and little endian as Network Byte
Order and Host Byte Order respectively. What is the difference between both
two?
- Usually in our host computer we store the data byte as Host Byte Order, for
- example, we store a integer in the RAM which might occupies 4 Byte, as Host
- Byte Order the higher Byte would be stored at the lower address of RAM, and
- the lower Byte would be stored at the higher address of RAM. However, contrast
- to this, Network Byte Order just take the totally opposite way to store the
- data, says, it will store the lower Byte at the lower address, and the higher
- Byte will stay at higher address.
-
- For the current communication of network, we normally exchange the information
- by surveying the data package, every two host wants to communicate with each
- other must send and receive data package through network. In order to maintain
- the identity of data through the transmission in the network, the order of the
- Byte storage must changed before sending and after receiving the data.
-
- There ten convenient functions to realize the conversion of Byte Order in
- GNUnet, as following:
- @table @asis
+Usually in our host computer we store the data byte as Host Byte Order, for
+example, we store a integer in the RAM which might occupies 4 Byte, as Host
+Byte Order the higher Byte would be stored at the lower address of RAM, and
+the lower Byte would be stored at the higher address of RAM. However, contrast
+to this, Network Byte Order just take the totally opposite way to store the
+data, says, it will store the lower Byte at the lower address, and the higher
+Byte will stay at higher address.
+
+For the current communication of network, we normally exchange the information
+by surveying the data package, every two host wants to communicate with each
+other must send and receive data package through network. In order to maintain
+the identity of data through the transmission in the network, the order of the
+Byte storage must changed before sending and after receiving the data.
+
+There ten convenient functions to realize the conversion of Byte Order in
+GNUnet, as following:
address@hidden @asis
@item uint16_t htons(uint16_t hostshort) Convert host byte order to net byte
order with short int
@@ -2873,22 +2783,10 @@ GNUNET_TIME_absolute_ntoh (struct
GNUNET_TIME_AbsoluteNBO a) Convert relative
time from network byte order.
@end table
address@hidden
address@hidden
-
-
address@hidden
-
address@hidden itemize
@c ***************************************************************************
address@hidden Cryptography API
address@hidden Cryptography API
address@hidden Cryptography API
@c %**end of header
address@hidden Top
-
-
-
The gnunetutil APIs provides the cryptographic primitives used in GNUnet.
GNUnet uses 2048 bit RSA keys for the session key exchange and for signing
messages by peers and most other public-key operations. Most researchers in
@@ -2919,22 +2817,11 @@ Finally, the crypto API provides a means to
deterministically generate a
used by most applications; most importantly,@
GNUNET_CRYPTO_rsa_key_create_from_hash does not create an RSA-key that should
be considered secure for traditional applications of RSA.
address@hidden
address@hidden
-
-
address@hidden
address@hidden itemize
@c ***************************************************************************
address@hidden Message Queue API
address@hidden Message Queue API
address@hidden Message Queue API
@c %**end of header
address@hidden Top
-
-
-
@strong{ Introduction }@ Often, applications need to queue messages that are to
be sent to other GNUnet peers, clients or services. As all of GNUnet's
message-based communication APIs, by design, do not allow messages to be
@@ -2990,24 +2877,25 @@ MQ provides macros to allocate an envelope containing a
message conveniently,@
automatically setting the size and type fields of the message.
Consider the following simple message, with the body consisting of a single
-number value.@ @code{} @examplestruct NumberMessage @{
+number value.@ @code{}
address@hidden
+struct NumberMessage @{
/** Type: GNUNET_MESSAGE_TYPE_EXAMPLE_1 */
- struct GNUNET_MessageHeader header; uint32_t number GNUNET_PACKED; @}; @end
- example
-
-
-
+ struct GNUNET_MessageHeader header; uint32_t number GNUNET_PACKED; @};
address@hidden example
An envelope containing an instance of the NumberMessage can be constructed like
-this: @examplestruct GNUNET_MQ_Envelope *ev; struct NumberMessage *msg; ev =
+this:
address@hidden
+struct GNUNET_MQ_Envelope *ev; struct NumberMessage *msg; ev =
GNUNET_MQ_msg (msg, GNUNET_MESSAGE_TYPE_EXAMPLE_1); msg->number = htonl (42);
@end example
- In the above code, @code{GNUNET_MQ_msg} is a macro. The return value is the
- newly allocated envelope. The first argument must be a pointer to some
- @code{struct} containing a @code{struct GNUNET_MessageHeader header} field,
- while the second argument is the desired message type, in host byte order.
+In the above code, @code{GNUNET_MQ_msg} is a macro. The return value is the
+newly allocated envelope. The first argument must be a pointer to some
address@hidden containing a @code{struct GNUNET_MessageHeader header} field,
+while the second argument is the desired message type, in host byte order.
The @code{msg} pointer now points to an allocated message, where the message
type and the message size are already set. The message's size is inferred from
@@ -3020,8 +2908,8 @@ allocated after the @code{msg} structure.
If no structure has been defined for the message,
@code{GNUNET_MQ_msg_header_extra} can be used to allocate additional space
-after the message header. The first argument then must be a pointer to a @code{
-GNUNET_MessageHeader}.
+after the message header. The first argument then must be a pointer to a
address@hidden
@strong{Envelope Properties}@ A few functions in MQ allow to set additional
properties on envelopes:
@@ -3052,22 +2940,11 @@ callback. When canceling an envelope, it is not
necessary@ to call
@code{GNUNET_MQ_discard}, and the envelope can't be sent again.
@strong{ Implementing Queues }@ @code{TODO}
address@hidden
address@hidden
-
-
address@hidden
address@hidden itemize
@c ***************************************************************************
address@hidden Service API
address@hidden Service API
address@hidden Service API
@c %**end of header
address@hidden Top
-
-
-
Most GNUnet code lives in the form of services. Services are processes that
offer an API for other components of the system to build on. Those other
components can be command-line tools for users, graphical user interfaces or
@@ -3129,21 +3006,10 @@ allowing existing 'normal' clients to set (possibly
persistent) statistic
values before terminating.
@end table
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
@c ***************************************************************************
address@hidden Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
address@hidden Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
address@hidden Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
@c %**end of header
address@hidden Top
-
-
-
A commonly used data structure in GNUnet is a (multi-)hash map. It is most
often used to map a peer identity to some data structure, but also to map
arbitrary keys to values (for example to track requests in the distributed hash
@@ -3152,42 +3018,41 @@ sometimes responsible for a large share of GNUnet's
overall memory consumption
(for some processes, 30% is not uncommon). The following text documents some
API quirks (and their implications for applications) that were recently
introduced to minimize the footprint of the hash map.
address@hidden
***************************************************************************
address@hidden Analysis
address@hidden Analysis
-
address@hidden %**end of header
-
address@hidden Top
-
-
- The main reason for the "excessive" memory consumption by the hash map is that
- GNUnet uses 512-bit cryptographic hash codes --- and the (multi-)hash map also
- uses the same 512-bit 'struct GNUNET_HashCode'. As a result, storing just the
- keys requires 64 bytes of memory for each key. As some applications like to
- keep a large number of entries in the hash map (after all, that's what maps
- are good for), 64 bytes per hash is significant: keeping a pointer to the
- value and having a linked list for collisions consume between 8 and 16 bytes,
- and 'malloc' may add about the same overhead per allocation, putting us in the
- 16 to 32 byte per entry ballpark. Adding a 64-byte key then triples the
- overall memory requirement for the hash map.
-
- To make things "worse", most of the time storing the key in the hash map is
- not required: it is typically already in memory elsewhere! In most cases, the
- values stored in the hash map are some application-specific struct that _also_
- contains the hash. Here is a simplified example: @examplestruct MyValue @{
- struct GNUNET_HashCode key; unsigned int my_data; @};
address@hidden
***************************************************************************
address@hidden Analysis
address@hidden %**end of header
+
+The main reason for the "excessive" memory consumption by the hash map is that
+GNUnet uses 512-bit cryptographic hash codes --- and the (multi-)hash map also
+uses the same 512-bit 'struct GNUNET_HashCode'. As a result, storing just the
+keys requires 64 bytes of memory for each key. As some applications like to
+keep a large number of entries in the hash map (after all, that's what maps
+are good for), 64 bytes per hash is significant: keeping a pointer to the
+value and having a linked list for collisions consume between 8 and 16 bytes,
+and 'malloc' may add about the same overhead per allocation, putting us in the
+16 to 32 byte per entry ballpark. Adding a 64-byte key then triples the
+overall memory requirement for the hash map.
+
+To make things "worse", most of the time storing the key in the hash map is
+not required: it is typically already in memory elsewhere! In most cases, the
+values stored in the hash map are some application-specific struct that _also_
+contains the hash. Here is a simplified example:
address@hidden
+struct MyValue @{
+struct GNUNET_HashCode key; unsigned int my_data; @};
// ...
val = GNUNET_malloc (sizeof (struct MyValue)); val->key = key; val->my_data =
-42; GNUNET_CONTAINER_multihashmap_put (map, &key, val, ...); @end example
+42; GNUNET_CONTAINER_multihashmap_put (map, &key, val, ...);
address@hidden example
This is a common pattern as later the entries might need to be removed, and at
that time it is convenient to have the key immediately at hand:
address@hidden (map, &val->key, val); @end
-example
address@hidden
+GNUNET_CONTAINER_multihashmap_remove (map, &val->key, val);
address@hidden example
Note that here we end up with two times 64 bytes for the key, plus maybe 64
@@ -3199,44 +3064,37 @@ those highlighted in this example. This is especially
true for maps with a
significant number of entries, as there we tend to really try to keep the
entries small.
@c ***************************************************************************
address@hidden Solution
address@hidden Solution
address@hidden %**end of header
-
address@hidden Top
-
-
-
- The solution that has now been implemented is to @strong{optionally} allow the
- hash map to not make a (deep) copy of the hash but instead have a pointer to
- the hash/key in the entry. This reduces the memory consumption for the key
- from 64 bytes to 4 to 8 bytes. However, it can also only work if the key is
- actually stored in the entry (which is the case most of the time) and if the
- entry does not modify the key (which in all of the code I'm aware of has been
- always the case if there key is stored in the entry). Finally, when the client
- stores an entry in the hash map, it @strong{must} provide a pointer to the key
- within the entry, not just a pointer to a transient location of the key. If
- the client code does not meet these requirements, the result is a dangling
- pointer and undefined behavior of the (multi-)hash map API.
address@hidden Solution
address@hidden %**end of header
+
+The solution that has now been implemented is to @strong{optionally} allow the
+hash map to not make a (deep) copy of the hash but instead have a pointer to
+the hash/key in the entry. This reduces the memory consumption for the key
+from 64 bytes to 4 to 8 bytes. However, it can also only work if the key is
+actually stored in the entry (which is the case most of the time) and if the
+entry does not modify the key (which in all of the code I'm aware of has been
+always the case if there key is stored in the entry). Finally, when the client
+stores an entry in the hash map, it @strong{must} provide a pointer to the key
+within the entry, not just a pointer to a transient location of the key. If
+the client code does not meet these requirements, the result is a dangling
+pointer and undefined behavior of the (multi-)hash map API.
@c ***************************************************************************
address@hidden Migration
address@hidden Migration
address@hidden Migration
@c %**end of header
address@hidden Top
-
-
-
- To use the new feature, first check that the values contain the respective key
- (and never modify it). Then, all calls to
- @code{GNUNET_CONTAINER_multihashmap_put} on the respective map must be audited
- and most likely changed to pass a pointer into the value's struct. For the
- initial example, the new code would look like this: @examplestruct MyValue @{
- struct GNUNET_HashCode key; unsigned int my_data; @};
+To use the new feature, first check that the values contain the respective key
+(and never modify it). Then, all calls to
address@hidden on the respective map must be audited
+and most likely changed to pass a pointer into the value's struct. For the
+initial example, the new code would look like this:
address@hidden
+struct MyValue @{
+struct GNUNET_HashCode key; unsigned int my_data; @};
// ...
val = GNUNET_malloc (sizeof (struct MyValue)); val->key = key; val->my_data =
-42; GNUNET_CONTAINER_multihashmap_put (map, &val->key, val, ...); @end example
+42; GNUNET_CONTAINER_multihashmap_put (map, &val->key, val, ...);
address@hidden example
Note that @code{&val} was changed to @code{&val->key} in the argument to the
@@ -3252,63 +3110,40 @@ map =
GNUNET_CONTAINER_multihashmap_create (SIZE, GNUNET_NO);
@end example
-
to
+
@example
map = GNUNET_CONTAINER_multihashmap_create (SIZE, GNUNET_YES);
@end example
-
-
If everything was done correctly, you now use about 60 bytes less memory per
entry in @code{map}. However, if now (or in the future) any call to @code{put}
does not ensure that the given key is valid until the entry is removed from the
map, undefined behavior is likely to be observed.
@c ***************************************************************************
address@hidden Conclusion
address@hidden Conclusion
address@hidden Conclusion
@c %**end of header
address@hidden Top
-
-
-
- The new optimization can is often applicable and can result in a reduction in
- memory consumption of up to 30% in practice. However, it makes the code less
- robust as additional invariants are imposed on the multi hash map client. Thus
- applications should refrain from enabling the new mode unless the resulting
- performance increase is deemed significant enough. In particular, it should
- generally not be used in new code (wait at least until benchmarks exist).
+The new optimization can is often applicable and can result in a reduction in
+memory consumption of up to 30% in practice. However, it makes the code less
+robust as additional invariants are imposed on the multi hash map client. Thus
+applications should refrain from enabling the new mode unless the resulting
+performance increase is deemed significant enough. In particular, it should
+generally not be used in new code (wait at least until benchmarks exist).
@c ***************************************************************************
address@hidden Availability
address@hidden Availability
address@hidden Availability
@c %**end of header
address@hidden Top
-
+The new multi hash map code was committed in SVN 24319 (will be in GNUnet
+0.9.4). Various subsystems (transport, core, dht, file-sharing) were
+previously audited and modified to take advantage of the new capability. In
+particular, memory consumption of the file-sharing service is expected to drop
+by 20-30% due to this change.
-
- The new multi hash map code was committed in SVN 24319 (will be in GNUnet
- 0.9.4). Various subsystems (transport, core, dht, file-sharing) were
- previously audited and modified to take advantage of the new capability. In
- particular, memory consumption of the file-sharing service is expected to drop
- by 20-30% due to this change.
address@hidden
address@hidden
-
-
address@hidden
-
address@hidden itemize
@c ***************************************************************************
address@hidden The CONTAINER_MDLL API
address@hidden The CONTAINER_MDLL API
address@hidden The CONTAINER_MDLL API
@c %**end of header
address@hidden Top
-
-
-
This text documents the GNUNET_CONTAINER_MDLL API. The GNUNET_CONTAINER_MDLL
API is similar to the GNUNET_CONTAINER_DLL API in that it provides operations
for the construction and manipulation of doubly-linked lists. The key
@@ -3327,23 +3162,29 @@ element may be in multiple doubly-linked lists, so we
cannot just have one
The solution is to have multiple fields that must have a name of the format
"next_XX" and "prev_XX" where "XX" is the name of one of the doubly-linked
-lists. Here is a simple example: @example struct MyMultiListElement @{ struct
+lists. Here is a simple example:
address@hidden
+struct MyMultiListElement @{ struct
MyMultiListElement *next_ALIST; struct MyMultiListElement *prev_ALIST; struct
MyMultiListElement *next_BLIST; struct MyMultiListElement *prev_BLIST; void
-*data; @}; @end example
+*data; @};
address@hidden example
Note that by convention, we use all-uppercase letters for the list names. In
addition, the program needs to have a location for the head and tail pointers
-for both lists, for example: @example static struct MyMultiListElement
+for both lists, for example:
address@hidden
+static struct MyMultiListElement
*head_ALIST; static struct MyMultiListElement *tail_ALIST; static struct
MyMultiListElement *head_BLIST; static struct MyMultiListElement *tail_BLIST;
@end example
-Using the MDLL-macros, we can now insert an element into the ALIST: @example
-GNUNET_CONTAINER_MDLL_insert (ALIST, head_ALIST, tail_ALIST, element); @end
-example
+Using the MDLL-macros, we can now insert an element into the ALIST:
address@hidden
+GNUNET_CONTAINER_MDLL_insert (ALIST, head_ALIST, tail_ALIST, element);
address@hidden example
Passing "ALIST" as the first argument to MDLL specifies which of the next/prev
@@ -3353,22 +3194,11 @@ differences between the MDDL and DLL-API. Like the
DLL-API, the MDLL-API offers
functions for inserting (at head, at tail, after a given element) and removing
elements from the list. Iterating over the list should be done by directly
accessing the "next_XX" and/or "prev_XX" members.
address@hidden
address@hidden
-
address@hidden
-
address@hidden itemize
@c ***************************************************************************
address@hidden The Automatic Restart Manager (ARM)
address@hidden The Automatic Restart Manager (ARM)
address@hidden The Automatic Restart Manager (ARM)
@c %**end of header
address@hidden Top
-
-
-
GNUnet's Automated Restart Manager (ARM) is the GNUnet service responsible for
system initialization and service babysitting. ARM starts and halts services,
detects configuration changes and restarts services impacted by the changes as
@@ -3377,17 +3207,12 @@ planned to incorporate automatic debugging for
diagnosing service crashes
providing developers insights about crash reasons. The purpose of this document
is to give GNUnet developer an idea about how ARM works and how to interact
with it.
+
@c ***************************************************************************
address@hidden Basic functionality
address@hidden Basic functionality
address@hidden Basic functionality
@c %**end of header
address@hidden Top
-
-
@itemize @bullet
-
-
@item ARM source code can be found under "src/arm".@ Service processes are
managed by the functions in "gnunet-service-arm.c" which is controlled with
"gnunet-arm.c" (main function in that file is ARM's entry point).
@@ -3405,21 +3230,10 @@ test_arm_api.c. The test case connects to ARM, starts
it, then uses it to start
a service "resolver", stops the "resolver" then stops "ARM".
@end itemize
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
@c ***************************************************************************
address@hidden Key configuration options
address@hidden Key configuration options
address@hidden Key configuration options
@c %**end of header
address@hidden Top
-
-
-
Configurations for ARM and services should be available in a .conf file (As an
example, see test_arm_api_data.conf). When running ARM, the configuration file
to use should be passed to the command:@ @code{@ $ gnunet-arm -s -c
@@ -3475,22 +3289,10 @@ that are going to run.@
@end table
address@hidden
address@hidden
-
-
address@hidden
-
address@hidden itemize
@c ***************************************************************************
address@hidden Availability
address@hidden Availability
address@hidden Availability
@c %**end of header
address@hidden Top
-
-
-
As mentioned before, one of the features provided by ARM is starting services
on demand. Consider the example of one service "client" that wants to connect
to another service a "server". The "client" will ask ARM to run the "server".
@@ -3525,18 +3327,8 @@ work).
@item Other client services now can directly connect directly to the "server".
@end itemize
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
@c ***************************************************************************
@node Reliability
address@hidden Reliability
-
address@hidden Top
-
-
One of the features provided by ARM, is the automatic restart of crashed
services.@ ARM needs to know which of the running services died. Function
@@ -3575,21 +3367,10 @@ backoff(S) will remain half an hour, hence ARM won't be
busy for a lot of time
trying to restart a problematic service.
@end itemize
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
@c ***************************************************************************
address@hidden GNUnet's TRANSPORT Subsystem
address@hidden GNUnet's TRANSPORT Subsystem
address@hidden GNUnet's TRANSPORT Subsystem
@c %**end of header
address@hidden Top
-
-
-
This chapter documents how the GNUnet transport subsystem works. The GNUnet
transport subsystem consists of three main components: the transport API (the
interface used by the rest of the system to access the transport service), the
@@ -3632,24 +3413,16 @@ clients
outbound traffic limits are enforced by CORE, not by us (!)
@item Enforce restrictions on P2P connection as specified by the blacklist
-configuration and blacklisting clients @end itemize
+configuration and blacklisting clients
address@hidden itemize
Note that the term "clients" in the list above really refers to the GNUnet-CORE
service, as CORE is typically the only client of the transport service.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden Address validation protocol
address@hidden Address validation protocol
@c %**end of header
address@hidden Top
-
-
-
This section documents how the GNUnet transport service validates connections
with other peers. It is a high-level description of the protocol necessary to
understand the details of the implementation. It should be noted that when we
@@ -3670,50 +3443,40 @@ claiming to be Alice). As a further complication, the
scheme has to work even
if say Alice is behind a NAT without traversal support and hence has no address
of her own (and thus Alice must always initiate the connection to Bob).
- An additional constraint is that HELLO messages do not contain a cryptographic
- signature since other peers must be able to edit (i.e. remove) addresses from
- the HELLO at any time (this was not true in GNUnet 0.8.x). A basic
- @strong{assumption} is that each peer knows the set of possible network
- addresses that it @strong{might} be reachable under (so for example, the
- external IP address of the NAT plus the LAN address(es) with the respective
- ports).
-
- The solution is the following. If Alice wants to validate that a given address
- for Bob is valid (i.e. is actually established @strong{directly} with the
- intended target), it sends a PING message over that connection to Bob. Note
- that in this case, Alice initiated the connection so only she knows which
- address was used for sure (Alice maybe behind NAT, so whatever address Bob
- sees may not be an address Alice knows she has). Bob checks that the address
- given in the PING is actually one of his addresses (does not belong to
- Mallory), and if it is, sends back a PONG (with a signature that says that Bob
- owns/uses the address from the PING). Alice checks the signature and is happy
- if it is valid and the address in the PONG is the address she used. This is
- similar to the 0.8.x protocol where the HELLO contained a signature from Bob
- for each address used by Bob. Here, the purpose code for the signature is
- @code{GNUNET_SIGNATURE_PURPOSE_TRANSPORT_PONG_OWN}. After this, Alice will
- remember Bob's address and consider the address valid for a while (12h in the
- current implementation). Note that after this exchange, Alice only considers
- Bob's address to be valid, the connection itself is not considered
- 'established'. In particular, Alice may have many addresses for Bob that she
- considers valid.
-
- The PONG message is protected with a nonce/challenge against replay attacks
- and uses an expiration time for the signature (but those are almost
- implementation details).
-
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden NAT library
+An additional constraint is that HELLO messages do not contain a cryptographic
+signature since other peers must be able to edit (i.e. remove) addresses from
+the HELLO at any time (this was not true in GNUnet 0.8.x). A basic
address@hidden is that each peer knows the set of possible network
+addresses that it @strong{might} be reachable under (so for example, the
+external IP address of the NAT plus the LAN address(es) with the respective
+ports).
+
+The solution is the following. If Alice wants to validate that a given address
+for Bob is valid (i.e. is actually established @strong{directly} with the
+intended target), it sends a PING message over that connection to Bob. Note
+that in this case, Alice initiated the connection so only she knows which
+address was used for sure (Alice maybe behind NAT, so whatever address Bob
+sees may not be an address Alice knows she has). Bob checks that the address
+given in the PING is actually one of his addresses (does not belong to
+Mallory), and if it is, sends back a PONG (with a signature that says that Bob
+owns/uses the address from the PING). Alice checks the signature and is happy
+if it is valid and the address in the PONG is the address she used. This is
+similar to the 0.8.x protocol where the HELLO contained a signature from Bob
+for each address used by Bob. Here, the purpose code for the signature is
address@hidden After this, Alice will
+remember Bob's address and consider the address valid for a while (12h in the
+current implementation). Note that after this exchange, Alice only considers
+Bob's address to be valid, the connection itself is not considered
+'established'. In particular, Alice may have many addresses for Bob that she
+considers valid.
+
+The PONG message is protected with a nonce/challenge against replay attacks
+and uses an expiration time for the signature (but those are almost
+implementation details).
+
address@hidden NAT library
@c %**end of header
address@hidden Top
-
-
-
The goal of the GNUnet NAT library is to provide a general-purpose API for NAT
traversal @strong{without} third-party support. So protocols that involve
contacting a third peer to help establish a connection between two peers are
@@ -3753,19 +3516,10 @@ works. The test function act as a local client,
initialize the NAT traversal
and then contact a @code{gnunet-nat-server} (running by default on
@code{gnunet.org}) and ask for a connection to be established. This way, it is
easy to test if the current NAT configuration is valid.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden Distance-Vector plugin
address@hidden Distance-Vector plugin
@c %**end of header
address@hidden Top
-
-
-
The Distance Vector (DV) transport is a transport mechanism that allows peers
to act as relays for each other, thereby connecting peers that would otherwise
be unable to connect. This gives a larger connection set to applications that
@@ -3818,119 +3572,78 @@ receives the messages, verifies both Alice and Carol
are known to Bob, and
re-wraps the message in a new DV message for Carol. The DV transport at Carol
receives this message, unwraps the original message, and delivers it to Carol
as though it came directly from Alice.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden SMTP plugin
address@hidden SMTP plugin
@c %**end of header
address@hidden Top
-
-
-
- This page describes the new SMTP transport plugin for GNUnet as it exists in
- the 0.7.x and 0.8.x branch. SMTP support is currently not available in GNUnet
- 0.9.x. This page also describes the transport layer abstraction (as it existed
- in 0.7.x and 0.8.x) in more detail and gives some benchmarking results. The
- performance results presented are quite old and maybe outdated at this point.
+This page describes the new SMTP transport plugin for GNUnet as it exists in
+the 0.7.x and 0.8.x branch. SMTP support is currently not available in GNUnet
+0.9.x. This page also describes the transport layer abstraction (as it existed
+in 0.7.x and 0.8.x) in more detail and gives some benchmarking results. The
+performance results presented are quite old and maybe outdated at this point.
@itemize @bullet
-
-
@item Why use SMTP for a peer-to-peer transport?
-
@item How does it work?
-
@item How do I configure my peer?
-
@item How do I test if it works?
-
@item How fast is it?
address@hidden Is there any additional documentation?
address@hidden itemize
address@hidden Is there any additional documentation? @end itemize
-
-
address@hidden Why use SMTP for a peer-to-peer transport?
address@hidden Why use SMTP for a peer-to-peer transport?
@c %**end of header
address@hidden Top
-
-
-
There are many reasons why one would not want to use SMTP:
@itemize @bullet
-
-
@item SMTP is using more bandwidth than TCP, UDP or HTTP
-
@item SMTP has a much higher latency.
-
@item SMTP requires significantly more computation (encoding and decoding time)
for the peers.
-
@item SMTP is significantly more complicated to configure.
-
@item SMTP may be abused by tricking GNUnet into sending mail to@
non-participating third parties.
@end itemize
-
So why would anybody want to use SMTP?
@itemize @bullet
-
-
@item SMTP can be used to contact peers behind NAT boxes (in virtual private
networks).
-
@item SMTP can be used to circumvent policies that limit or prohibit
peer-to-peer traffic by masking as "legitimate" traffic.
-
@item SMTP uses E-mail addresses which are independent of a specific IP, which
can be useful to address peers that use dynamic IP addresses.
-
@item SMTP can be used to initiate a connection (e.g. initial address exchange)
and peers can then negotiate the use of a more efficient protocol (e.g. TCP)
for the actual communication.
@end itemize
+In summary, SMTP can for example be used to send a message to a peer behind a
+NAT box that has a dynamic IP to tell the peer to establish a TCP connection
+to a peer outside of the private network. Even an extraordinary overhead for
+this first message would be irrelevant in this type of situation.
- In summary, SMTP can for example be used to send a message to a peer behind a
- NAT box that has a dynamic IP to tell the peer to establish a TCP connection
- to a peer outside of the private network. Even an extraordinary overhead for
- this first message would be irrelevant in this type of situation.
-
address@hidden How does it work?
address@hidden How does it work?
@c %**end of header
address@hidden Top
-
-
-
- When a GNUnet peer needs to send a message to another GNUnet peer that has
- advertised (only) an SMTP transport address, GNUnet base64-encodes the message
- and sends it in an E-mail to the advertised address. The advertisement
- contains a filter which is placed in the E-mail header, such that the
- receiving host can filter the tagged E-mails and forward it to the GNUnet peer
- process. The filter can be specified individually by each peer and be changed
- over time. This makes it impossible to censor GNUnet E-mail messages by
- searching for a generic filter.
+When a GNUnet peer needs to send a message to another GNUnet peer that has
+advertised (only) an SMTP transport address, GNUnet base64-encodes the message
+and sends it in an E-mail to the advertised address. The advertisement
+contains a filter which is placed in the E-mail header, such that the
+receiving host can filter the tagged E-mails and forward it to the GNUnet peer
+process. The filter can be specified individually by each peer and be changed
+over time. This makes it impossible to censor GNUnet E-mail messages by
+searching for a generic filter.
address@hidden How do I configure my peer?
address@hidden How do I configure my peer?
@c %**end of header
address@hidden Top
-
-
-
- First, you need to configure @code{procmail} to filter your inbound E-mail for
- GNUnet traffic. The GNUnet messages must be delivered into a pipe, for example
- @code{/tmp/gnunet.smtp}. You also need to define a filter that is used by
- procmail to detect GNUnet messages. You are free to choose whichever filter
- you like, but you should make sure that it does not occur in your other
- E-mail. In our example, we will use @code{X-mailer: GNUnet}. The
- @code{~/.procmailrc} configuration file then looks like this:
+First, you need to configure @code{procmail} to filter your inbound E-mail for
+GNUnet traffic. The GNUnet messages must be delivered into a pipe, for example
address@hidden/tmp/gnunet.smtp}. You also need to define a filter that is used
by
+procmail to detect GNUnet messages. You are free to choose whichever filter
+you like, but you should make sure that it does not occur in your other
+E-mail. In our example, we will use @code{X-mailer: GNUnet}. The
address@hidden/.procmailrc} configuration file then looks like this:
@example
:0:
* ^X-mailer: GNUnet
@@ -3939,81 +3652,74 @@ for the actual communication.
:0: /var/spool/mail/
@end example
+After adding this file, first make sure that your regular E-mail still works
+(e.g. by sending an E-mail to yourself). Then edit the GNUnet configuration.
+In the section @code{SMTP} you need to specify your E-mail address under
address@hidden, your mail server (for outgoing mail) under @code{SERVER}, the
+filter (X-mailer: GNUnet in the example) under @code{FILTER} and the name of
+the pipe under @code{PIPE}.@ The completed section could then look like this:
address@hidden
+EMAIL = me@@mail.gnu.org MTU = 65000 SERVER = mail.gnu.org:25 FILTER =
+"X-mailer: GNUnet" PIPE = /tmp/gnunet.smtp
address@hidden example
- After adding this file, first make sure that your regular E-mail still works
- (e.g. by sending an E-mail to yourself). Then edit the GNUnet configuration.
- In the section @code{SMTP} you need to specify your E-mail address under
- @code{EMAIL}, your mail server (for outgoing mail) under @code{SERVER}, the
- filter (X-mailer: GNUnet in the example) under @code{FILTER} and the name of
- the pipe under @code{PIPE}.@ The completed section could then look like this:
- @exampleEMAIL = me@@mail.gnu.org MTU = 65000 SERVER = mail.gnu.org:25 FILTER =
- "X-mailer: GNUnet" PIPE = /tmp/address@hidden example
-
-
- Finally, you need to add @code{smtp} to the list of @code{TRANSPORTS} in the
- @code{GNUNETD} section. GNUnet peers will use the E-mail address that you
- specified to contact your peer until the advertisement times out. Thus, if you
- are not sure if everything works properly or if you are not planning to be
- online for a long time, you may want to configure this timeout to be short,
- e.g. just one hour. For this, set @code{HELLOEXPIRES} to @code{1} in the
- @code{GNUNETD} section.
+Finally, you need to add @code{smtp} to the list of @code{TRANSPORTS} in the
address@hidden section. GNUnet peers will use the E-mail address that you
+specified to contact your peer until the advertisement times out. Thus, if you
+are not sure if everything works properly or if you are not planning to be
+online for a long time, you may want to configure this timeout to be short,
+e.g. just one hour. For this, set @code{HELLOEXPIRES} to @code{1} in the
address@hidden section.
- This should be it, but you may probably want to test it first.@
address@hidden How do I test if it works?
+This should be it, but you may probably want to test it first.@
address@hidden How do I test if it works?
@c %**end of header
address@hidden Top
-
-
-
- Any transport can be subjected to some rudimentary tests using the
- @code{gnunet-transport-check} tool. The tool sends a message to the local node
- via the transport and checks that a valid message is received. While this test
- does not involve other peers and can not check if firewalls or other network
- obstacles prohibit proper operation, this is a great testcase for the SMTP
- transport since it tests pretty much nearly all of the functionality.
+Any transport can be subjected to some rudimentary tests using the
address@hidden tool. The tool sends a message to the local node
+via the transport and checks that a valid message is received. While this test
+does not involve other peers and can not check if firewalls or other network
+obstacles prohibit proper operation, this is a great testcase for the SMTP
+transport since it tests pretty much nearly all of the functionality.
@code{gnunet-transport-check} should only be used without running
@code{gnunetd} at the same time. By default, @code{gnunet-transport-check}
tests all transports that are specified in the configuration file. But you can
specifically test SMTP by giving the option @code{--transport=smtp}.
- Note that this test always checks if a transport can receive and send. While
- you can configure most transports to only receive or only send messages, this
- test will only work if you have configured the transport to send and receive
- messages.
-
address@hidden How fast is it?
address@hidden %**end of header
-
address@hidden Top
-
-
-
- We have measured the performance of the UDP, TCP and SMTP transport layer
- directly and when used from an application using the GNUnet core. Measureing
- just the transport layer gives the better view of the actual overhead of the
- protocol, whereas evaluating the transport from the application puts the
- overhead into perspective from a practical point of view.
-
- The loopback measurements of the SMTP transport were performed on three
- different machines spanning a range of modern SMTP configurations. We used a
- PIII-800 running RedHat 7.3 with the Purdue Computer Science configuration
- which includes filters for spam. We also used a Xenon 2 GHZ with a vanilla
- RedHat 8.0 sendmail configuration. Furthermore, we used qmail on a PIII-1000
- running Sorcerer GNU Linux (SGL).@ The numbers for UDP and TCP are provided
- using the SGL configuration. The qmail benchmark uses qmailâs internal
- filtering whereas the sendmail benchmarks relies on procmail to filter and
- deliver the mail. We used the transport layer to send a message of b bytes
- (excluding transport protocol headers) directly to the local machine. This
- way, network latency and packet loss on the wire have no impact on the
- timings. n messages were sent sequentially over the transport layer, sending
- message i+1 after the i-th message was received. All messages were sent over
- the same connection and the time to establish the connection was not taken
- into account since this overhead is miniscule in practice --- as long as a
- connection is used for a significant number of messages. @multitable
- @columnfractions 0.166666666666667 0.166666666666667 0.166666666666667
- 0.166666666666667 0.166666666666667 0.166666666666667
+Note that this test always checks if a transport can receive and send. While
+you can configure most transports to only receive or only send messages, this
+test will only work if you have configured the transport to send and receive
+messages.
+
address@hidden How fast is it?
address@hidden %**end of header
+
+We have measured the performance of the UDP, TCP and SMTP transport layer
+directly and when used from an application using the GNUnet core. Measureing
+just the transport layer gives the better view of the actual overhead of the
+protocol, whereas evaluating the transport from the application puts the
+overhead into perspective from a practical point of view.
+
+The loopback measurements of the SMTP transport were performed on three
+different machines spanning a range of modern SMTP configurations. We used a
+PIII-800 running RedHat 7.3 with the Purdue Computer Science configuration
+which includes filters for spam. We also used a Xenon 2 GHZ with a vanilla
+RedHat 8.0 sendmail configuration. Furthermore, we used qmail on a PIII-1000
+running Sorcerer GNU Linux (SGL).@ The numbers for UDP and TCP are provided
+using the SGL configuration. The qmail benchmark uses qmail's internal
+filtering whereas the sendmail benchmarks relies on procmail to filter and
+deliver the mail. We used the transport layer to send a message of b bytes
+(excluding transport protocol headers) directly to the local machine. This
+way, network latency and packet loss on the wire have no impact on the
+timings. n messages were sent sequentially over the transport layer, sending
+message i+1 after the i-th message was received. All messages were sent over
+the same connection and the time to establish the connection was not taken
+into account since this overhead is miniscule in practice --- as long as a
+connection is used for a significant number of messages.
address@hidden
address@hidden 0.166666666666667 0.166666666666667 0.166666666666667
+0.166666666666667 0.166666666666667 0.166666666666667
@item Transport
@@ -4066,142 +3772,104 @@ specifically test SMTP by giving the option
@code{--transport=smtp}.
@end multitable
- The benchmarks show that UDP and TCP are, as expected, both significantly
- faster compared with any of the SMTP services. Among the SMTP implementations,
- there can be significant differences depending on the SMTP configuration.
- Filtering with an external tool like procmail that needs to re-parse its
- configuration for each mail can be very expensive. Applying spam filters can
- also significantly impact the performance of the underlying SMTP
- implementation. The microbenchmark shows that SMTP can be a viable solution
- for initiating peer-to-peer sessions: a couple of seconds to connect to a peer
- are probably not even going to be noticed by users. The next benchmark
- measures the possible throughput for a transport. Throughput can be measured
- by sending multiple messages in parallel and measuring packet loss. Note that
- not only UDP but also the TCP transport can actually loose messages since the
- TCP implementation drops messages if the @code{write} to the socket would
- block. While the SMTP protocol never drops messages itself, it is often so
- slow that only a fraction of the messages can be sent and received in the
- given time-bounds. For this benchmark we report the message loss after
- allowing t time for sending m messages. If messages were not sent (or
- received) after an overall timeout of t, they were considered lost. The
- benchmark was performed using two Xeon 2 GHZ machines running RedHat 8.0 with
- sendmail. The machines were connected with a direct 100 MBit ethernet
- connection.@ Figures udp1200, tcp1200 and smtp-MTUs show that the throughput
- for messages of size 1,200 octects is 2,343 kbps, 3,310 kbps and 6 kbps for
- UDP, TCP and SMTP respectively. The high per-message overhead of SMTP can be
- improved by increasing the MTU, for example, an MTU of 12,000 octets improves
- the throughput to 13 kbps as figure smtp-MTUs shows. Our research paper) has
- some more details on the benchmarking results.
+The benchmarks show that UDP and TCP are, as expected, both significantly
+faster compared with any of the SMTP services. Among the SMTP implementations,
+there can be significant differences depending on the SMTP configuration.
+Filtering with an external tool like procmail that needs to re-parse its
+configuration for each mail can be very expensive. Applying spam filters can
+also significantly impact the performance of the underlying SMTP
+implementation. The microbenchmark shows that SMTP can be a viable solution
+for initiating peer-to-peer sessions: a couple of seconds to connect to a peer
+are probably not even going to be noticed by users. The next benchmark
+measures the possible throughput for a transport. Throughput can be measured
+by sending multiple messages in parallel and measuring packet loss. Note that
+not only UDP but also the TCP transport can actually loose messages since the
+TCP implementation drops messages if the @code{write} to the socket would
+block. While the SMTP protocol never drops messages itself, it is often so
+slow that only a fraction of the messages can be sent and received in the
+given time-bounds. For this benchmark we report the message loss after
+allowing t time for sending m messages. If messages were not sent (or
+received) after an overall timeout of t, they were considered lost. The
+benchmark was performed using two Xeon 2 GHZ machines running RedHat 8.0 with
+sendmail. The machines were connected with a direct 100 MBit ethernet
+connection.@ Figures udp1200, tcp1200 and smtp-MTUs show that the throughput
+for messages of size 1,200 octects is 2,343 kbps, 3,310 kbps and 6 kbps for
+UDP, TCP and SMTP respectively. The high per-message overhead of SMTP can be
+improved by increasing the MTU, for example, an MTU of 12,000 octets improves
+the throughput to 13 kbps as figure smtp-MTUs shows. Our research paper) has
+some more details on the benchmarking results.
+
address@hidden Bluetooth plugin
address@hidden %**end of header
+
+This page describes the new Bluetooth transport plugin for GNUnet. The plugin
+is still in the testing stage so don't expect it to work perfectly. If you
+have any questions or problems just post them here or ask on the IRC channel.
@itemize @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden Bluetooth plugin
address@hidden %**end of header
-
address@hidden Top
-
-
-
- This page describes the new Bluetooth transport plugin for GNUnet. The plugin
- is still in the testing stage so don't expect it to work perfectly. If you
- have any questions or problems just post them here or ask on the IRC channel.
address@hidden @bullet
-
-
@item What do I need to use the Bluetooth plugin transport?
-
@item How does it work?
-
@item What possible errors should I be aware of?
-
@item How do I configure my peer?
-
@item How can I test it?
@end itemize
-
address@hidden What do I need to use the Bluetooth plugin transport?
address@hidden What do I need to use the Bluetooth plugin transport?
@c %**end of header
+If you are a Linux user and you want to use the Bluetooth transport plugin you
+should install the BlueZ development libraries (if they aren't already
+installed). For instructions about how to install the libraries you should
+check out the BlueZ site (@uref{http://www.bluez.org/, http://www.bluez.org}).
+If you don't know if you have the necesarry libraries, don't worry, just run
+the GNUnet configure script and you will be able to see a notification at the
+end which will warn you if you don't have the necessary libraries.
address@hidden Top
-
+If you are a Windows user you should have installed the
address@hidden/@emph{MSys2} with the latest updates (especially the
address@hidden header). If this is your first build of GNUnet on Windows you
+should check out the SBuild repository. It will semi-automatically assembles a
address@hidden/@emph{MSys2} installation with a lot of extra packages which are
+needed for the GNUnet build. So this will ease your work!@ Finally you just
+have to be sure that you have the correct drivers for your Bluetooth device
+installed and that your device is on and in a discoverable mode. The Windows
+Bluetooth Stack supports only the RFCOMM protocol so we cannot turn on your
+device programatically!
-
- If you are a Linux user and you want to use the Bluetooth transport plugin you
- should install the BlueZ development libraries (if they aren't already
- installed). For instructions about how to install the libraries you should
- check out the BlueZ site (@uref{http://www.bluez.org/, http://www.bluez.org}).
- If you don't know if you have the necesarry libraries, don't worry, just run
- the GNUnet configure script and you will be able to see a notification at the
- end which will warn you if you don't have the necessary libraries.
-
- If you are a Windows user you should have installed the
- @emph{MinGW}/@emph{MSys2} with the latest updates (especially the
- @emph{ws2bth} header). If this is your first build of GNUnet on Windows you
- should check out the SBuild repository. It will semi-automatically assembles a
- @emph{MinGW}/@emph{MSys2} installation with a lot of extra packages which are
- needed for the GNUnet build. So this will ease your work!@ Finally you just
- have to be sure that you have the correct drivers for your Bluetooth device
- installed and that your device is on and in a discoverable mode. The Windows
- Bluetooth Stack supports only the RFCOMM protocol so we cannot turn on your
- device programatically!
-
address@hidden How does it work?
address@hidden How does it work?
@c %**end of header
address@hidden Top
-
+The Bluetooth transport plugin uses virtually the same code as the WLAN plugin
+and only the helper binary is different. The helper takes a single argument,
+which represents the interface name and is specified in the configuration
+file. Here are the basic steps that are followed by the helper binary used on
+Linux:
-
- The Bluetooth transport plugin uses virtually the same code as the WLAN plugin
- and only the helper binary is different. The helper takes a single argument,
- which represents the interface name and is specified in the configuration
- file. Here are the basic steps that are followed by the helper binary used on
- Linux:
@itemize @bullet
-
-
@item it verifies if the name corresponds to a Bluetooth interface name
-
@item it verifies if the iterface is up (if it is not, it tries to bring it up)
-
@item it tries to enable the page and inquiry scan in order to make the device
discoverable and to accept incoming connection requests
-
@emph{The above operations require root access so you should start the
transport plugin with root privileges.}
-
@item it finds an available port number and registers a SDP service which will
be used to find out on which port number is the server listening on and switch
the socket in listening mode
-
@item it sends a HELLO message with its address
-
@item finally it forwards traffic from the reading sockets to the STDOUT and
-from the STDIN to the writing socket @end itemize
-
+from the STDIN to the writing socket
address@hidden itemize
- Once in a while the device will make an inquiry scan to discover the nearby
- devices and it will send them randomly HELLO messages for peer discovery.
+Once in a while the device will make an inquiry scan to discover the nearby
+devices and it will send them randomly HELLO messages for peer discovery.
address@hidden What possible errors should I be aware of?
address@hidden What possible errors should I be aware of?
@c %**end of header
address@hidden Top
-
-
-
@emph{This section is dedicated for Linux users}
- Well there are many ways in which things could go wrong but I will try to
- present some tools that you could use to debug and some scenarios.
address@hidden
address@hidden
-
+Well there are many ways in which things could go wrong but I will try to
+present some tools that you could use to debug and some scenarios.
address@hidden @bullet
@item @code{bluetoothd -n -d} : use this command to enable logging in the
foreground and to print the logging messages
@@ -4226,51 +3894,46 @@ as provider.
@item @code{hcitool} : another useful tool which can be used to configure the
device and to send some particular commands to it.
address@hidden @code{hcidump} : could be used for low level debugging @end
itemize
-
address@hidden @code{hcidump} : could be used for low level debugging
address@hidden itemize
address@hidden How do I configure my peer?
address@hidden How do I configure my peer?
@c %**end of header
address@hidden Top
-
-
+On Linux, you just have to be sure that the interface name corresponds to the
+one that you want to use. Use the @code{hciconfig} tool to check that. By
+default it is set to hci0 but you can change it.
- On Linux, you just have to be sure that the interface name corresponds to the
- one that you want to use. Use the @code{hciconfig} tool to check that. By
- default it is set to hci0 but you can change it.
-
- A basic configuration looks like this : @example[transport-bluetooth]
+A basic configuration looks like this:
address@hidden
+[transport-bluetooth]
# Name of the interface (typically hciX)
INTERFACE = hci0
# Real hardware, no testing
-TESTMODE = 0 TESTING_IGNORE_KEYS = ACCEPT_FROM; @end example
-
+TESTMODE = 0 TESTING_IGNORE_KEYS = ACCEPT_FROM;
address@hidden example
- In order to use the Bluetooth transport plugin when the transport service is
- started, you must add the plugin name to the default transport service plugins
- list. For example: @example[transport] ... PLUGINS = dns bluetooth ... @end
- example
+In order to use the Bluetooth transport plugin when the transport service is
+started, you must add the plugin name to the default transport service plugins
+list. For example:
address@hidden
+[transport] ... PLUGINS = dns bluetooth ...
address@hidden example
If you want to use only the Bluetooth plugin set @emph{PLUGINS = bluetooth}
- On Windows, you cannot specify which device to use. The only thing that you
- should do is to add @emph{bluetooth} on the plugins list of the transport
- service.
+On Windows, you cannot specify which device to use. The only thing that you
+should do is to add @emph{bluetooth} on the plugins list of the transport
+service.
address@hidden How can I test it?
address@hidden How can I test it?
@c %**end of header
address@hidden Top
-
-
-
- If you have two Bluetooth devices on the same machine which use Linux you
- must:
+If you have two Bluetooth devices on the same machine which use Linux you
+must:
@itemize @bullet
-
@item create two different file configuration (one which will use the first
interface (@emph{hci0}) and the other which will use the second interface
(@emph{hci1})). Let's name them @emph{peer1.conf} and @emph{peer2.conf}.
@@ -4300,68 +3963,46 @@ some conflicts between the Bluetooth stacks. (At least
that is what happend on
my machine : I wasn't able to use the Bluesoleil stack and the WINDCOMM one in
the same time).
- If you have two different machines and your configuration files are good you
- can use the same scenario presented on the begining of this section.
-
- Another way to test the plugin functionality is to create your own application
- which will use the GNUnet framework with the Bluetooth transport service.
address@hidden @bullet
+If you have two different machines and your configuration files are good you
+can use the same scenario presented on the begining of this section.
+Another way to test the plugin functionality is to create your own application
+which will use the GNUnet framework with the Bluetooth transport service.
address@hidden
-
address@hidden itemize
address@hidden The implementation of the Bluetooth transport plugin
address@hidden The implementation of the Bluetooth transport plugin
@c %**end of header
address@hidden Top
-
-
+This page describes the implementation of the Bluetooth transport plugin.
- This page describes the implementation of the Bluetooth transport plugin.
+First I want to remind you that the Bluetooth transport plugin uses virtually
+the same code as the WLAN plugin and only the helper binary is different. Also
+the scope of the helper binary from the Bluetooth transport plugin is the same
+as the one used for the wlan transport plugin: it acceses the interface and
+then it forwards traffic in both directions between the Bluetooth interface
+and stdin/stdout of the process involved.
- First I want to remind you that the Bluetooth transport plugin uses virtually
- the same code as the WLAN plugin and only the helper binary is different. Also
- the scope of the helper binary from the Bluetooth transport plugin is the same
- as the one used for the wlan transport plugin: it acceses the interface and
- then it forwards traffic in both directions between the Bluetooth interface
- and stdin/stdout of the process involved.
-
- The Bluetooth plugin transport could be used both on Linux and Windows
- platforms.
+The Bluetooth plugin transport could be used both on Linux and Windows
+platforms.
@itemize @bullet
-
-
@item Linux functionality
-
@item Windows functionality
address@hidden Pending Features
address@hidden itemize
address@hidden Pending Features @end itemize
-
-
address@hidden Linux functionality
address@hidden Linux functionality
@c %**end of header
address@hidden Top
-
-
-
- In order to implement the plugin functionality on Linux I used the BlueZ
- stack. For the communication with the other devices I used the RFCOMM
- protocol. Also I used the HCI protocol to gain some control over the device.
- The helper binary takes a single argument (the name of the Bluetooth
- interface) and is separated in two stages:
address@hidden @bullet
address@hidden THE INITIALIZATION
+In order to implement the plugin functionality on Linux I used the BlueZ
+stack. For the communication with the other devices I used the RFCOMM
+protocol. Also I used the HCI protocol to gain some control over the device.
+The helper binary takes a single argument (the name of the Bluetooth
+interface) and is separated in two stages:
address@hidden address@hidden @bullet
address@hidden THE INITIALIZATION
@c %**end of header
-
address@hidden Top
-
-
@itemize @bullet
-
@item first, it checks if we have root privilegies (@emph{Remember that we need
to have root privilegies in order to be able to bring the interface up if it is
down or to change its state.} ).
@@ -4377,40 +4018,26 @@ interface:}
On the @emph{open_device} method:
@itemize @bullet
-
@item creates a HCI socket used to send control events to the the device
-
@item searches for the device ID using the interface name
-
@item saves the device MAC address
-
@item checks if the interface is down and tries to bring it UP
-
@item checks if the interface is in discoverable mode and tries to make it
discoverable
-
@item closes the HCI socket and binds the RFCOMM one
-
@item switches the RFCOMM socket in listening mode
-
@item registers the SDP service (the service will be used by the other devices
-to get the port on which this device is listening on) @end itemize
-
+to get the port on which this device is listening on)
address@hidden itemize
@item drops the root privilegies
@strong{If the interface is not a Bluetooth interface the helper exits with a
suitable error}
@end itemize
address@hidden THE LOOP
address@hidden THE LOOP
@c %**end of header
address@hidden Top
-
-
-
-
-
The helper binary uses a list where it saves all the connected neighbour
devices (@emph{neighbours.devices}) and two buffers (@emph{write_pout} and
@emph{write_std}). The first message which is send is a control message with
@@ -4433,7 +4060,8 @@ the connection and saves the socket on a list with the
reading sockets.
@item Every time when it receives something from a reading socket it parses the
message, verifies the CRC and saves it in the @emph{write_std} buffer in order
-to be sent later to the STDOUT. @end itemize
+to be sent later to the STDOUT.
address@hidden itemize
So in the main loop we use the select function to wait until one of the file
@@ -4456,13 +4084,9 @@ another one when trying to send a message it will
attempt one more time. If it
fails again, then it skips the message.}@ @emph{Also you should know that the
transport Bluetooth plugin has support for @strong{broadcast messages}.}
address@hidden Detailes about the broadcast implementation
address@hidden Detailes about the broadcast implementation
@c %**end of header
address@hidden Top
-
-
-
First I want to point out that the broadcast functionality for the CONTROL
messages is not implemented in a conventional way. Since the inquiry scan time
is too big@ and it will take some time to send a message to all the
@@ -4470,7 +4094,6 @@ discoverable devices I decided to tackle the problem in a
different way. Here
is how I did it:
@itemize @bullet
-
@item If it is the first time when I have to broadcast a message I make an
inquiry scan and save all the devices' addresses to a vector.
@@ -4489,7 +4112,6 @@ list and send messages to the devices saved in it.
@strong{Therefore} :
@itemize @bullet
-
@item every time when I have a broadcast message I look up on the list for a
new device and send the message to it
@@ -4513,34 +4135,29 @@ on the broadcast list to see if we are connected to
that device. If not we try
to connect to it and in case of success we save the address and the socket on
the list. If we are already connected to that device we simply use the socket.
@end itemize
-
@end itemize
address@hidden Windows functionality
address@hidden Windows functionality
@c %**end of header
address@hidden Top
-
+For Windows I decided to use the Microsoft Bluetooth stack which has the
+advantage of coming standard from Windows XP SP2. The main disadvantage is
+that it only supports the RFCOMM protocol so we will not be able to have a low
+level control over the Bluetooth device. Therefore it is the user
+responsability to check if the device is up and in the discoverable mode. Also
+there are no tools which could be used for debugging in order to read the data
+coming from and going to a Bluetooth device, which obviously hindered my work.
+Another thing that slowed down the implementation of the plugin (besides that
+I wasn't too accomodated with the win32 API) was that there were some bugs on
+MinGW regarding the Bluetooth. Now they are solved but you should keep in mind
+that you should have the latest updates (especially the @emph{ws2bth} header).
-
- For Windows I decided to use the Microsoft Bluetooth stack which has the
- advantage of coming standard from Windows XP SP2. The main disadvantage is
- that it only supports the RFCOMM protocol so we will not be able to have a low
- level control over the Bluetooth device. Therefore it is the user
- responsability to check if the device is up and in the discoverable mode. Also
- there are no tools which could be used for debugging in order to read the data
- coming from and going to a Bluetooth device, which obviously hindered my work.
- Another thing that slowed down the implementation of the plugin (besides that
- I wasn't too accomodated with the win32 API) was that there were some bugs on
- MinGW regarding the Bluetooth. Now they are solved but you should keep in mind
- that you should have the latest updates (especially the @emph{ws2bth} header).
-
- Besides the fact that it uses the Windows Sockets, the Windows implemenation
- follows the same principles as the Linux one:
+Besides the fact that it uses the Windows Sockets, the Windows implemenation
+follows the same principles as the Linux one:
@itemize @bullet
-
address@hidden
It has a initalization part where it initializes the Windows Sockets, creates a
RFCOMM socket which will be binded and switched to the listening mode and
registers a SDP service.
@@ -4548,84 +4165,62 @@ registers a SDP service.
In the Microsoft Bluetooth API there are two ways to work with the SDP:
@itemize @bullet
-
@item an easy way which works with very simple service records
@item a hard way which is useful when you need to update or to delete the
record
@end itemize
address@hidden itemize
+Since I only needed the SDP service to find out on which port the device is
+listening on and that did not change, I decided to use the easy way. In order
+to register the service I used the @emph{WSASetService} function and I
+generated the @emph{Universally Unique Identifier} with the @emph{guidgen.exe}
+Windows's tool.
- Since I only needed the SDP service to find out on which port the device is
- listening on and that did not change, I decided to use the easy way. In order
- to register the service I used the @emph{WSASetService} function and I
- generated the @emph{Universally Unique Identifier} with the @emph{guidgen.exe}
- Windows's tool.
-
- In the loop section the only difference from the Linux implementation is that
- I used the GNUNET_NETWORK library for functions like @emph{accept},
- @emph{bind}, @emph{connect} or @emph{select}. I decided to use the
- GNUNET_NETWORK library because I also needed to interact with the STDIN and
- STDOUT handles and on Windows the select function is only defined for sockets,
- and it will not work for arbitrary file handles. @end itemize
+In the loop section the only difference from the Linux implementation is that
+I used the GNUNET_NETWORK library for functions like @emph{accept},
address@hidden, @emph{connect} or @emph{select}. I decided to use the
+GNUNET_NETWORK library because I also needed to interact with the STDIN and
+STDOUT handles and on Windows the select function is only defined for sockets,
+and it will not work for arbitrary file handles. @end itemize
- Another difference between Linux and Windows implementation is that in Linux,
- the Bluetooth address is represented in 48 bits while in Windows is
- represented in 64 bits. Therefore I had to do some changes on
- @emph{plugin_transport_wlan} header.
+Another difference between Linux and Windows implementation is that in Linux,
+the Bluetooth address is represented in 48 bits while in Windows is
+represented in 64 bits. Therefore I had to do some changes on
address@hidden header.
- Also, currently on Windows the Bluetooth plugin doesn't have support for
- broadcast messages. When it receives a broadcast message it will skip it.
+Also, currently on Windows the Bluetooth plugin doesn't have support for
+broadcast messages. When it receives a broadcast message it will skip it.
address@hidden Pending features
address@hidden Pending features
@c %**end of header
address@hidden Top
-
-
@itemize @bullet
-
@item Implement the broadcast functionality on Windows @emph{(currently working
on)}
@item Implement a testcase for the helper :@ @emph{@ The testcase consists of a
program which emaluates the plugin and uses the helper. It will simulate
-connections, disconnections and data transfers.@ } @end itemize
-
-
- If you have a new idea about a feature of the plugin or suggestions about how
- I could improve the implementation you are welcome to comment or to contact
- me.
- @itemize @bullet
-
-
address@hidden
-
+connections, disconnections and data transfers.@ }
@end itemize
address@hidden WLAN plugin
address@hidden %**end of header
address@hidden Top
+If you have a new idea about a feature of the plugin or suggestions about how
+I could improve the implementation you are welcome to comment or to contact
+me.
address@hidden WLAN plugin
address@hidden %**end of header
This section documents how the wlan transport plugin works. Parts which are not
implemented yet or could be better implemented are described at the end.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden The ATS Subsystem
address@hidden The ATS Subsystem
@c %**end of header
address@hidden Top
-
-
-
ATS stands for "automatic transport selection", and the function of ATS in
GNUnet is to decide on which address (and thus transport plugin) should be used
for two peers to communicate, and what bandwidth limits should be imposed on
@@ -4642,27 +4237,16 @@ make the actual allocation decision. This last critical
step is left to an ATS
plugin, as we have implemented (currently three) different allocation
strategies which differ significantly in their performance and maturity, and it
is still unclear if any particular plugin is generally superior.
address@hidden
address@hidden
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's CORE Subsystem
address@hidden GNUnet's CORE Subsystem
@c %**end of header
address@hidden Top
-
-
-
- The CORE subsystem in GNUnet is responsible for securing link-layer
- communications between nodes in the GNUnet overlay network. CORE builds on the
- TRANSPORT subsystem which provides for the actual, insecure, unreliable
- link-layer communication (for example, via UDP or WLAN), and then adds
- fundamental security to the connections:
- @itemize @bullet
-
+The CORE subsystem in GNUnet is responsible for securing link-layer
+communications between nodes in the GNUnet overlay network. CORE builds on the
+TRANSPORT subsystem which provides for the actual, insecure, unreliable
+link-layer communication (for example, via UDP or WLAN), and then adds
+fundamental security to the connections:
address@hidden @bullet
@item confidentiality with so-called perfect forward secrecy; we use
@uref{http://en.wikipedia.org/wiki/Elliptic_curve_Diffie%E2%80%93Hellman,
@@ -4686,165 +4270,139 @@ keys)
@item liveness (keep-alive messages, timeout)
@end itemize
address@hidden Limitations
address@hidden %**end of header
-
address@hidden Top
-
-
-
- CORE does not perform @uref{http://en.wikipedia.org/wiki/Routing, routing};
- using CORE it is only possible to communicate with peers that happen to
- already be "directly" connected with each other. CORE also does not have an
- API to allow applications to establish such "direct" connections --- for this,
- applications can ask TRANSPORT, but TRANSPORT might not be able to establish a
- "direct" connection. The TOPOLOGY subsystem is responsible for trying to keep
- a few "direct" connections open at all times. Applications that need to talk
- to particular peers should use the CADET subsystem, as it can establish
- arbitrary "indirect" connections.
-
- Because CORE does not perform routing, CORE must only be used directly by
- applications that either perform their own routing logic (such as anonymous
- file-sharing) or that do not require routing, for example because they are
- based on flooding the network. CORE communication is unreliable and delivery
- is possibly out-of-order. Applications that require reliable communication
- should use the CADET service. Each application can only queue one message per
- target peer with the CORE service at any time; messages cannot be larger than
- approximately 63 kilobytes. If messages are small, CORE may group multiple
- messages (possibly from different applications) prior to encryption. If
- permitted by the application (using the @uref{http://baus.net/on-tcp_cork/,
- cork} option), CORE may delay transmissions to facilitate grouping of multiple
- small messages. If cork is not enabled, CORE will transmit the message as soon
- as TRANSPORT allows it (TRANSPORT is responsible for limiting bandwidth and
- congestion control). CORE does not allow flow control; applications are
- expected to process messages at line-speed. If flow control is needed,
- applications should use the CADET service.
address@hidden When is a peer "connected"?
address@hidden %**end of header
-
address@hidden Top
-
-
-
- In addition to the security features mentioned above, CORE also provides one
- additional key feature to applications using it, and that is a limited form of
- protocol-compatibility checking. CORE distinguishes between TRANSPORT-level
- connections (which enable communication with other peers) and
- application-level connections. Applications using the CORE API will
- (typically) learn about application-level connections from CORE, and not about
- TRANSPORT-level connections. When a typical application uses CORE, it will
- specify a set of message types (from @code{gnunet_protocols.h}) that it
- understands. CORE will then notify the application about connections it has
- with other peers if and only if those applications registered an intersecting
- set of message types with their CORE service. Thus, it is quite possible that
- CORE only exposes a subset of the established direct connections to a
- particular application --- and different applications running above CORE might
- see different sets of connections at the same time.
-
- A special case are applications that do not register a handler for any message
- type. CORE assumes that these applications merely want to monitor connections
- (or "all" messages via other callbacks) and will notify those applications
- about all connections. This is used, for example, by the @code{gnunet-core}
- command-line tool to display the active connections. Note that it is also
- possible that the TRANSPORT service has more active connections than the CORE
- service, as the CORE service first has to perform a key exchange with
- connecting peers before exchanging information about supported message types
- and notifying applications about the new connection.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetcore
address@hidden %**end of header
-
address@hidden Top
-
-
-
- The CORE API (defined in @code{gnunet_core_service.h}) is the basic messaging
- API used by P2P applications built using GNUnet. It provides applications the
- ability to send and receive encrypted messages to the peer's "directly"
- connected neighbours.
-
- As CORE connections are generally "direct" connections,@ applications must not
- assume that they can connect to arbitrary peers this way, as "direct"
- connections may not always be possible. Applications using CORE are notified
- about which peers are connected. Creating new "direct" connections must be
- done using the TRANSPORT API.
-
- The CORE API provides unreliable, out-of-order delivery. While the
- implementation tries to ensure timely, in-order delivery, both message losses
- and reordering are not detected and must be tolerated by the application. Most
- important, the core will NOT perform retransmission if messages could not be
- delivered.
-
- Note that CORE allows applications to queue one message per connected peer.
- The rate at which each connection operates is influenced by the preferences
- expressed by local application as well as restrictions imposed by the other
- peer. Local applications can express their preferences for particular
- connections using the "performance" API of the ATS service.
-
- Applications that require more sophisticated transmission capabilities such as
- TCP-like behavior, or if you intend to send messages to arbitrary remote
- peers, should use the CADET API.
-
- The typical use of the CORE API is to connect to the CORE service using
- @code{GNUNET_CORE_connect}, process events from the CORE service (such as
- peers connecting, peers disconnecting and incoming messages) and send messages
- to connected peers using @code{GNUNET_CORE_notify_transmit_ready}. Note that
- applications must cancel pending transmission requests if they receive a
- disconnect event for a peer that had a transmission pending; furthermore,
- queueing more than one transmission request per peer per application using the
- service is not permitted.
-
- The CORE API also allows applications to monitor all communications of the
- peer prior to encryption (for outgoing messages) or after decryption (for
- incoming messages). This can be useful for debugging, diagnostics or to
- establish the presence of cover traffic (for anonymity). As monitoring
- applications are often not interested in the payload, the monitoring callbacks
- can be configured to only provide the message headers (including the message
- type and size) instead of copying the full data stream to the monitoring
- client.
-
- The init callback of the @code{GNUNET_CORE_connect} function is called with
- the hash of the public key of the peer. This public key is used to identify
- the peer globally in the GNUnet network. Applications are encouraged to check
- that the provided hash matches the hash that they are using (as theoretically
- the application may be using a different configuration file with a different
- private key, which would result in hard to find bugs).
-
- As with most service APIs, the CORE API isolates applications from crashes of
- the CORE service. If the CORE service crashes, the application will see
- disconnect events for all existing connections. Once the connections are
- re-established, the applications will be receive matching connect events.
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden The CORE Client-Service Protocol
address@hidden %**end of header
-
address@hidden Top
-
-
-
- This section describes the protocol between an application using the CORE
- service (the client) and the CORE service process itself. @settitle Setup @c
- %**end of header
-
address@hidden Top
-
-
-
- When a client connects to the CORE service, it first sends a
- @code{InitMessage} which specifies options for the connection and a set of
- message type values which are supported by the application. The options
- bitmask specifies which events the client would like to be notified about. The
- options include:
address@hidden Limitations
address@hidden %**end of header
+
+CORE does not perform @uref{http://en.wikipedia.org/wiki/Routing, routing};
+using CORE it is only possible to communicate with peers that happen to
+already be "directly" connected with each other. CORE also does not have an
+API to allow applications to establish such "direct" connections --- for this,
+applications can ask TRANSPORT, but TRANSPORT might not be able to establish a
+"direct" connection. The TOPOLOGY subsystem is responsible for trying to keep
+a few "direct" connections open at all times. Applications that need to talk
+to particular peers should use the CADET subsystem, as it can establish
+arbitrary "indirect" connections.
+
+Because CORE does not perform routing, CORE must only be used directly by
+applications that either perform their own routing logic (such as anonymous
+file-sharing) or that do not require routing, for example because they are
+based on flooding the network. CORE communication is unreliable and delivery
+is possibly out-of-order. Applications that require reliable communication
+should use the CADET service. Each application can only queue one message per
+target peer with the CORE service at any time; messages cannot be larger than
+approximately 63 kilobytes. If messages are small, CORE may group multiple
+messages (possibly from different applications) prior to encryption. If
+permitted by the application (using the @uref{http://baus.net/on-tcp_cork/,
+cork} option), CORE may delay transmissions to facilitate grouping of multiple
+small messages. If cork is not enabled, CORE will transmit the message as soon
+as TRANSPORT allows it (TRANSPORT is responsible for limiting bandwidth and
+congestion control). CORE does not allow flow control; applications are
+expected to process messages at line-speed. If flow control is needed,
+applications should use the CADET service.
+
address@hidden When is a peer "connected"?
address@hidden %**end of header
+
+In addition to the security features mentioned above, CORE also provides one
+additional key feature to applications using it, and that is a limited form of
+protocol-compatibility checking. CORE distinguishes between TRANSPORT-level
+connections (which enable communication with other peers) and
+application-level connections. Applications using the CORE API will
+(typically) learn about application-level connections from CORE, and not about
+TRANSPORT-level connections. When a typical application uses CORE, it will
+specify a set of message types (from @code{gnunet_protocols.h}) that it
+understands. CORE will then notify the application about connections it has
+with other peers if and only if those applications registered an intersecting
+set of message types with their CORE service. Thus, it is quite possible that
+CORE only exposes a subset of the established direct connections to a
+particular application --- and different applications running above CORE might
+see different sets of connections at the same time.
+
+A special case are applications that do not register a handler for any message
+type. CORE assumes that these applications merely want to monitor connections
+(or "all" messages via other callbacks) and will notify those applications
+about all connections. This is used, for example, by the @code{gnunet-core}
+command-line tool to display the active connections. Note that it is also
+possible that the TRANSPORT service has more active connections than the CORE
+service, as the CORE service first has to perform a key exchange with
+connecting peers before exchanging information about supported message types
+and notifying applications about the new connection.
+
address@hidden libgnunetcore
address@hidden %**end of header
+
+The CORE API (defined in @code{gnunet_core_service.h}) is the basic messaging
+API used by P2P applications built using GNUnet. It provides applications the
+ability to send and receive encrypted messages to the peer's "directly"
+connected neighbours.
+
+As CORE connections are generally "direct" connections,@ applications must not
+assume that they can connect to arbitrary peers this way, as "direct"
+connections may not always be possible. Applications using CORE are notified
+about which peers are connected. Creating new "direct" connections must be
+done using the TRANSPORT API.
+
+The CORE API provides unreliable, out-of-order delivery. While the
+implementation tries to ensure timely, in-order delivery, both message losses
+and reordering are not detected and must be tolerated by the application. Most
+important, the core will NOT perform retransmission if messages could not be
+delivered.
+
+Note that CORE allows applications to queue one message per connected peer.
+The rate at which each connection operates is influenced by the preferences
+expressed by local application as well as restrictions imposed by the other
+peer. Local applications can express their preferences for particular
+connections using the "performance" API of the ATS service.
+
+Applications that require more sophisticated transmission capabilities such as
+TCP-like behavior, or if you intend to send messages to arbitrary remote
+peers, should use the CADET API.
+
+The typical use of the CORE API is to connect to the CORE service using
address@hidden, process events from the CORE service (such as
+peers connecting, peers disconnecting and incoming messages) and send messages
+to connected peers using @code{GNUNET_CORE_notify_transmit_ready}. Note that
+applications must cancel pending transmission requests if they receive a
+disconnect event for a peer that had a transmission pending; furthermore,
+queueing more than one transmission request per peer per application using the
+service is not permitted.
+
+The CORE API also allows applications to monitor all communications of the
+peer prior to encryption (for outgoing messages) or after decryption (for
+incoming messages). This can be useful for debugging, diagnostics or to
+establish the presence of cover traffic (for anonymity). As monitoring
+applications are often not interested in the payload, the monitoring callbacks
+can be configured to only provide the message headers (including the message
+type and size) instead of copying the full data stream to the monitoring
+client.
+
+The init callback of the @code{GNUNET_CORE_connect} function is called with
+the hash of the public key of the peer. This public key is used to identify
+the peer globally in the GNUnet network. Applications are encouraged to check
+that the provided hash matches the hash that they are using (as theoretically
+the application may be using a different configuration file with a different
+private key, which would result in hard to find bugs).
+
+As with most service APIs, the CORE API isolates applications from crashes of
+the CORE service. If the CORE service crashes, the application will see
+disconnect events for all existing connections. Once the connections are
+re-established, the applications will be receive matching connect events.
+
address@hidden The CORE Client-Service Protocol
address@hidden %**end of header
+
+This section describes the protocol between an application using the CORE
+service (the client) and the CORE service process itself.
+
address@hidden Setup
address@hidden %**end of header
+
+When a client connects to the CORE service, it first sends a
address@hidden which specifies options for the connection and a set of
+message type values which are supported by the application. The options
+bitmask specifies which events the client would like to be notified about. The
+options include:
@table @asis
@item GNUNET_CORE_OPTION_NOTHING No notifications
@@ -4865,65 +4423,46 @@ messages
Typical applications will only monitor for connection status changes.
- The CORE service responds to the @code{InitMessage} with an
- @code{InitReplyMessage} which contains the peer's identity. Afterwards, both
- CORE and the client can send messages.
address@hidden Notifications
+The CORE service responds to the @code{InitMessage} with an
address@hidden which contains the peer's identity. Afterwards, both
+CORE and the client can send messages.
address@hidden Notifications
@c %**end of header
+The CORE will send @code{ConnectNotifyMessage}s and
address@hidden whenever peers connect or disconnect from the
+CORE (assuming their type maps overlap with the message types registered by
+the client). When the CORE receives a message that matches the set of message
+types specified during the @code{InitMessage} (or if monitoring is enabled in
+for inbound messages in the options), it sends a @code{NotifyTrafficMessage}
+with the peer identity of the sender and the decrypted payload. The same
+message format (except with @code{GNUNET_MESSAGE_TYPE_CORE_NOTIFY_OUTBOUND}
+for the message type) is used to notify clients monitoring outbound messages;
+here, the peer identity given is that of the receiver.
address@hidden Top
-
-
-
- The CORE will send @code{ConnectNotifyMessage}s and
- @code{DisconnectNotifyMessage}s whenever peers connect or disconnect from the
- CORE (assuming their type maps overlap with the message types registered by
- the client). When the CORE receives a message that matches the set of message
- types specified during the @code{InitMessage} (or if monitoring is enabled in
- for inbound messages in the options), it sends a @code{NotifyTrafficMessage}
- with the peer identity of the sender and the decrypted payload. The same
- message format (except with @code{GNUNET_MESSAGE_TYPE_CORE_NOTIFY_OUTBOUND}
- for the message type) is used to notify clients monitoring outbound messages;
- here, the peer identity given is that of the receiver.
address@hidden Sending
address@hidden Sending
@c %**end of header
address@hidden Top
-
-
-
- When a client wants to transmit a message, it first requests a transmission
- slot by sending a @code{SendMessageRequest} which specifies the priority,
- deadline and size of the message. Note that these values may be ignored by
- CORE. When CORE is ready for the message, it answers with a
- @code{SendMessageReady} response. The client can then transmit the payload
- with a @code{SendMessage} message. Note that the actual message size in the
- @code{SendMessage} is allowed to be smaller than the size in the original
- request. A client may at any time send a fresh @code{SendMessageRequest},
- which then superceeds the previous @code{SendMessageRequest}, which is then no
- longer valid. The client can tell which @code{SendMessageRequest} the CORE
- service's @code{SendMessageReady} message is for as all of these messages
- contain a "unique" request ID (based on a counter incremented by the client
- for each request).
address@hidden @bullet
-
-
address@hidden
+When a client wants to transmit a message, it first requests a transmission
+slot by sending a @code{SendMessageRequest} which specifies the priority,
+deadline and size of the message. Note that these values may be ignored by
+CORE. When CORE is ready for the message, it answers with a
address@hidden response. The client can then transmit the payload
+with a @code{SendMessage} message. Note that the actual message size in the
address@hidden is allowed to be smaller than the size in the original
+request. A client may at any time send a fresh @code{SendMessageRequest},
+which then superceeds the previous @code{SendMessageRequest}, which is then no
+longer valid. The client can tell which @code{SendMessageRequest} the CORE
+service's @code{SendMessageReady} message is for as all of these messages
+contain a "unique" request ID (based on a counter incremented by the client
+for each request).
address@hidden itemize
address@hidden The CORE Peer-to-Peer Protocol
address@hidden The CORE Peer-to-Peer Protocol
@c %**end of header
address@hidden Top
-
address@hidden Creating the EphemeralKeyMessage
address@hidden Creating the EphemeralKeyMessage
@c %**end of header
address@hidden Top
-
-
-
When the CORE service starts, each peer creates a fresh ephemeral (ECC)
public-private key pair and signs the corresponding @code{EphemeralKeyMessage}
with its long-term key (which we usually call the peer's identity; the hash of
--
To stop receiving notification emails like this one, please contact
address@hidden