[GNUnet-SVN] r25676 - gnunet-java

[gnunet]

Author: dold
Date: 2013-01-03 01:51:57 +0100 (Thu, 03 Jan 2013)
New Revision: 25676

Modified:
   gnunet-java/ISSUES
Log:
issues


Modified: gnunet-java/ISSUES
===================================================================
--- gnunet-java/ISSUES  2013-01-03 00:43:57 UTC (rev 25675)
+++ gnunet-java/ISSUES  2013-01-03 00:51:57 UTC (rev 25676)
@@ -1,375 +1,53 @@
+== meeting January 3, 2013 ==
 
-* in a real network:
- * what do we do if an authority is not responsive during key generation?
- * there's no way to prove that to other authorities
- * maybe send a "i tried to send message <msg> to authority n, it didn't work, 
please try yourself"
-   to the other authorities
- * => if it works, great, if it doesn't, other authorities are convinced that 
the offending authority is down
+* https://gnunet.org/bugs/view.php?id=2720
+ * had no idea where to look for this
+ * non-deterministic
+ * "pkill -9 gnunet" gets annoying ;)
 
- * numbering authorities:
-  * non-participating authority's pubkey is g^0 = 1
-  * what if the voting supervisor sends the authorities fake numbers?
-   * e.g. authority A_1 gets another numbering than A_2
-   * authorities should just check with each other
+* rounds:
+ 1. every peer broadcasts its set difference estimator
+ 2. every peer responds to incoming difference estimators with its ibf of 
appropriate size
+  * but max. one response per round per other peer
+  * should we do this naively, or coalesce them as much as possible?
+ 3. every peer responds to the IBF with the actual elements (not hashes) that 
only he has,
+    as well as a list of hashes, where he needs the actual elements.
+ 4. receive the actual elements
+* how do we determine the time for one round?
 
- * complaints:
-  * during key share verification
-   * offending authorities are excluded by the cooperating peers
-  * reporting the pubkey to the supervisor
-  * during vote casting
-   * voter should be notified
-  * during tally counting
+* how reliable of an indicator is an empty strata estimator?
+ * bad, when we have large sets, there will always be overflows in the high 
frequency strata
+ 
+* benchmarks / measurements of quality for consensus
+ * how do we judge the quality of consensus implementation for challenging 
situations?
 
+* GNUNET_MESH_peer_request_connect_add, can it fail, how can it fail, what is 
the timeout (see docs)?
 
-* documentation: how should I describe the whole voting process?
+* in mesh, tunnels are two-way. each peer has a broadcast tunnel, and incoming 
tunnels from other peers.
+ * now there are two ways to send a message to another peer, which of the 
tunnels is preferable / does is even matter?
+  * i.e., is there a performance difference in to-origin vs. unicast?
 
-* next step in the implementation: signing
-* what should we use for signing by the authorities?
- * gnunet uses RSA-2048 (?), probably makes sense to use the same here
+* the number of possible groups is very large, shouldn't we restrict their 
number?
+ * shouldn't we introduce this as another parameter for conclude?
+* assume every peer in our session is either muted or in agreement, how do we 
proceed with generating the groups?
+ * how do we generate/rank the groups?
 
-* there are approx. 4-5 different kinds of more or less publicly available 
keys, we should agree on a terminology
- * sigma_i
- * w_i : TallyKey
- * h_i
- * h : group pubkey
- * authority pubkey for signatures
+* interpretation of infinity as timeout for conclude: only stop exchaning once 
we have perfect consensus
 
+* what is there to say about broadcast performance in gerneral?
+* bart talked about NSE style broadcast, what is this?
 
-* why is the commitment to the *public* key even necessary?
+* Jan 02 22:03:52-356968 core-api-22511 INFO (Re)connecting to CORE service, 
monitoring messages of type 0
+ * how does that make sense?
 
-* style question: punctuation and displayed equations
+* IBF hash function: there's no need to have a cryptographic hash, as no 
randomization is possible,
+  and the domain of the hash function actually is really small. To find one 
collision on an IBF with n buckets, on average
+  we only have to try n random values, there's no need to find a SHA-256 
collision
 
------------------------------------------------------------------
+* when the client violates the protocol, what should the service do?
+ * most services use GNUNET_break?
+ * which is not very helful, as developers for other bindings (e.g. java ;) 
have to look up the right line in the c code,
+   with no error message
 
+* GNUNET_APPLICATION_TYPE_END obvious, but not documented
 
-* coverty
- * there will be many unread field warnings for the construct messages
-
-
-
-* what is pogen.sh, and why do I have to run it sometimes / why doesn't make 
do this?
-
-* next steps in the simulation?
- * probably switch from the synchronous implementation to async
-  * use mesh right away? or a simpler communication simulation, based on 
scheduler
-
-* how's the progress in testbed/testing?
-
-* peer vs user:
- * authority should be a peer (as an authority has to be reachable in the 
network)
- * i don't see any reason for the voter to be a peer
-  * only has to send ballot to an authority *somehow*, can be mesh, web 
interface, ...
-
-* about the ballot dissemination:
- * 
http://redlinernotes.com/docs/Classics/Epidemic%20Algorithms%20for%20Replicated%20Database%20Maintenance.pdf
-   (Epidemic algorithms for replicated database maintenance)
-   turned out to be the most useful of the papers
- * rumor mongering
-  * new votes are "hot" (=being spread) until enough other authorities already 
know about it
- * anti-entropy => randomly selected pair of authorities resoves their 
differences
-  * optimization: recent-update list gets sent first, then only hash is 
compared
-  * used to keep rumor mongering going
- * after voting period:
-   * naive solution: anti-entropy has to be executed for all authority pairs
-   * probably better:
-    * a few authorities pull all votes
-    * ... and distribute them.
-    * could use signatures, so only one authority has to collect all ballots
-     * receivers of the collected ballots only check signatures, no more 
communication costs
- * will be challenging to select and test the right parameters
-
-
---------------------------------------------------------------
-
-some relevant reconciliation/consensus papers:
-https://gnunet.org/node/2014
-https://gnunet.org/node/2015
-
-
-* discuss consent API
- * how is a consent group formed?
-
-* how much of C++ should we use?
- * only to interface NTL or also for the implementation?
-
-* existing recon implementation
- * compiles now, with make, not yet autotools (I'm not very familiar with 
autotools, yet!, yet!))
- * prob_reconcile and prob_recon_support seem very useful (implementing the 
basic set reconciliation)
- * reconciled_set implements partitioning (useful => expected linear time), 
-   interleaved with socket operations
-
-
-* organizing test cases
- * no support for JUnit mostly
-
-* not really relevant right now:
-    * discuss PKIs
-     * I don't really know where to start there
-     * ePA would be quite interesing, but again, where to start?
-     * WoT:
-      * GPG itself has no API, but there's 
http://www.gnupg.org/related_software/gpgme/
-     * whole PKI in java http://www.ejbca.org/download.html
-
-
-
-----------------------------------------------------------
-
-* currently trying to rule out a heisenbug in gnunet-java
- * turns up only sometimes, never when running tests individually
- * stacktraces are just wrong:
-
-java.lang.AssertionError: notifyTransmitReady called on disconnected client
-       at org.gnunet.util.Client.notifyTransmitReady(Client.java:152)
-       at 
org.gnunet.requests.RequestQueue.handleNextTransmit(RequestQueue.java:102)
-       at org.gnunet.requests.RequestQueue.add(RequestQueue.java:158)
-       at org.gnunet.statistics.Statistics.set(Statistics.java:277)
-       at org.gnunet.statistics.StatisticsTest$17.run(StatisticsTest.java:263)
-       at org.gnunet.util.Scheduler$TaskConfiguration.run(Scheduler.java:200)
-       at 
org.gnunet.util.Scheduler$TaskConfiguration.access$1100(Scheduler.java:115)
-       at org.gnunet.util.Scheduler.runReady(Scheduler.java:505)
-       at org.gnunet.util.Scheduler.run(Scheduler.java:481)
-       at org.gnunet.util.Scheduler.run(Scheduler.java:447)
-       at org.gnunet.nse.NSETest.test_nse(NSETest.java:52)
-       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
-       at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
-       at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
-       at java.lang.reflect.Method.invoke(Method.java:616)
-
-the NSE-test case somehow calls statistics ... that would be only possible if
-the tasks got mixed up, e.g. because of static variables in Scheduler
-
-
-=> Confirmed, test cases, even from different classes, get called in the same 
environment!
-
-
-
-
-about the release last week:
-actually gnunet-java-0.9.4.jar should have the name 
gnunet-java-installer-0.9.4, as there would
-be confusion with the library gnunet-java.jar
-
-
-* consensus:
- * how do we handle the "endgame" phase of a consensus session?
-  * clients can call conclude(), no new values will be accepted until enough 
peers
-    have been reconciled with
- * how do we manage memory/values for reconciled values?
-  * memory does not seem sufficient if we want to use a file / database / etc.
- * should consensus itself handle timestamps of set elements?
-  * conclude() would need that
- * deletion of elements should not be supported, should it?
-  * would be possible with anti-messages, see the gossiping overview paper
- * handling multiple peers: when to partition?
-  * i.e. sometimes we can't satisfy 
- * scheduling of exchanges / when should we reconcile with whom?
-
-
-* should consensus have a fixed bytestring length, or handle variable lengths?
- * probably the former as it is simpler
- * later could be done by just reconciling hashes, not the data itself
-
-* should there be the ability to remove/add peer to a consensus session?
- * I don't think so
-
-* what size should the consensus_id be?
-
-
-* how should the command line tool work?
- * either we introduce sessions with local UIDs, or we have to provide the 
list of peers every time
-
-
-* more literature:
- * http://www.ics.uci.edu/~eppstein/pubs/EppGooUye-SIGCOMM-11.pdf
-  * more recent (2011), they are aware of the minsky/trachtenberg CPI approach
-   * but only compare themselves with the BASIC-RECON approach, not 
PARTITION-RECON
-  * claim to resolve differences between two hosts in one round,
-    with computational and communication costs linear in the size of the symm. 
difference
-  * does an estimate of set difference size with the first message => not very 
nice for p2p
-  * make extensive use of invertible bloom filters
- * but no public implementation available :(
-
- * http://www.ics.uci.edu/~eppstein/pubs/EppGooUye-SIGCOMM-11.pdf
-  * also from 2011, estimates difference size with bloom filters, uses CPI, 
and is still computationally expensive
-
-
-
-* found a TR with the same title by minsky/trachtenberg, also add it to biblio?
- * http://dspace.library.cornell.edu/bitstream/1813/5788/1/2000-1796.pdf
-
-
-transport:
-* offer_hello: why a task, not a callback?
-* blacklist: why do we pass a config handle, not a transport handle?
-
-
-
-* automake stuff:
- * INCLUDES is deprecated:
-   http://www.gnu.org/software/automake/manual/automake.html#Program-Variables
-   INCLUDES
-   This does the same job as AM_CPPFLAGS (or any per-target _CPPFLAGS variable 
if it is used). It is an older name for the same functionality. This variable 
is deprecated; we suggest using AM_CPPFLAGS and per-target _CPPFLAGS instead. 
-
-------------------------------------------
-
-stumbling block: make check uses the service binary in the installed version.
-
-my bug: consensus service does not receive the CLIENT_JOIN message
-due to logging output of util, the client is accepted by the server, the 
client sends the message (notify transmit ready handler is called)
-but the message does not arrive at the service.
-
-why does this not work (in test_consensus_api):
-GNUNET_log_setup ("test_consensus_api", "DEBUG", NULL);
-GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "hi\n");
-(outputs nothing)
-
-* how do I use gdb with the test wrappers?
-
-* test wrapper: if the test wrapper segfaults, I have to pkill the peer 
manually.
-
-make test_consensus_api does not recreate libgnunetconsensus.la when 
consensus_api.c has changed
-
-warnings after executing tests
-  Nov 21 20:36:07-102628 util-27239 WARNING RSA signature verification failed 
at crypto_rsa.c:1349: Bad signature
-  Nov 21 20:36:07-102676 transport-27239 WARNING External protocol violation 
detected at gnunet-service-transport_validation.c:1152.
-
-* what happens on consensus disconnect? should the api handle this? *can* the 
api even handle this?
- * lost updates etc., should actually just re-create consensus
-
-* in general, what happens if e.g. during CONSENSUS_insert the connection to 
the consensus service drops?
- * the api can't really do anything, because the elements are not stored in 
the api, I suggest calling
-   new_element_cb with NULL
-
-
-* make inclusion of the local peer optional? or is this too ugly?
- * include: starting a session to test consensus locally would require 
knowledge of local PeerIdentity
- * exclude: starting consensus with a list of peers would require filtering 
the local peer
- * => compromise, always works
-
-* the initial_elements is a bit non-symmetric
- * should the api send them in the CLIENT_JOIN message or just do some 
CLIENT_INSERTs?
- * wouldn't it be nicer to just use inserts and have a CONSENSUS_begin() to 
signal that there is no more available local data
- * with some methods of storage, initial elements would not be in the required 
format anyway (e.g. DB)
-
-
-* sending elements to the service: should we do something like bart's ack / 
receive windows?
-
-* how do we prevent a "late submission" of a value, which would sabotage 
consensus?
- * by building a consensus group step by step, and asking all the members of 
the existing
-   consensus group if they also have the update from the peer new to the group
-   => use some commitment protocol (2PC/3PC/etc.)
- * if members of the old group do not respond timely to the suggestion to the 
new peer, we leave
-   that group, and join a new one with the new peer (and all cooperating peers 
from the old group)
- * commitment protocols require timeouts. how do we choose/compute/adapt sane 
values for that?
-
-* what would be a canonical example for being in two consensuses?
- * can this even happen with the above suggestion?
-
-* Fischer, Lynch and Paterson have an important impossibility result for 
consensus.
- * according to "Stumbling over consensus research: Misunderstandings and 
issues" 
http://research.microsoft.com/en-us/people/aguilera/stumbling-chapter.pdf
-   it is often misunderstood / does not matter that much in practice
-
--------------------------------------------------------------------------------------------
-
-* the strata estimator's "inspitarion:: 
http://www.mathcs.emory.edu/~cheung/papers/StreamDB/Probab/1985-Flajolet-Probabilistic-counting.pdf
- * maybe we can apply the same analysis techniques?
-
-* what about byte order and hashcode?
-
-* what happens if only one local client wants to conclude, and another still 
sends values?  when is a peer concluding?
- * when all its clients want to conclude? when one wants to conclude?
- * for voting, multiple clients on one authority does not really make sens, 
does it?
- * we could treat client as (peer_id, cli_id), but wouldn't that be too 
complicated / unncecessary?
-
-* stumbling block: why does TESTING_start_peer's config need the [autostart] 
section with DEFAULTSERVICES
-  to start core, but consensus service is started? how does arm know it should 
start consensus, but does not start core, even with AUTOSTART=YES?
-
-* would use / tried to use notification ctx, but there's no way to kill a 
client once it's in a context without killing the whole context.
-
-* how do i get the local peer's peer identity?
- * core? => done
-
-* are duplicate elements allowed to be passed to the new element callback? or 
should we guarantee that every distinct element only gets passed once?
- * duplicate-freeness is hard to guarantee, assume client sents value to 
service, at the same time service receives the same value over the network
-   and sends it to the client
- * => just allow duplicates, but try to minimize them in an efficient 
implementation
-
-* quite general question: what prevents a peer from just sending huge amounts 
of data / invalid ballots?
-
-* i don't get why multihashmap is implemented the way it is. Am I right that 
it does actually only use 4 byte (unsigned int) of the 16-bit hash?
-  is sha-2 really a great hash function for this?
- * I guess this is for historical reasons (to assoc. data with peers) but got 
used for other purposes?
-
-* have been looking at GNUNET_BloomFilter. Should IBF also be implemented with 
on-disk structures?
-
-* hash functions. what hash functions should the IBF use?
-
-
-------------------------------------------------------------------------
-
-GNUNET_CRYPTO_hash and GNUNET_CRYPTO_xor:
- * target and destination alias to the same memory location?
-
-* strage behavior with fgets and ctrl+c/ctrl+d
-
-* problem: what if difference digest does not fit in one message?
-
-* while connecting to a service, CTRL+c only cancels one connection try, gives 
up after 50 of them
- * is this intended?
-
-* GNUNET_TESTBED_controller_start
- * documentation of IP param, IP vs hostname, why both?
- * create peer can't take NULL host for localhost, inconsistent?
-    LOG testbed-api-hosts-2669 WARNING Host with id: 0 already created
-    LOG testbed-api-2669 WARNING Treating NULL host as localhost. Multiple 
references to localhost may break when localhost freed before calling 
disconnect 
-
-the format if a hostfile for testbed profile does not seem documented
-
-TESTBED_create does not seem to be implemented?
-
-* there is no GNUNET_TESTBED_Operation_cancel
-
-* when something (e.g. the controller status cb) has something missing in the 
docs, should i add it, or
-  contact the author?
-
-would use TESTBED_run but how do I get to the peers? continuation is a task, 
not a testbed master
- * => no TestRunContext available
-currently using TESTBED_test_run with PROGRAM_run2
-
-* args of run: shouldn't it say somewhere that it's NULL-terminated? why is it 
null-terminated?
-
-* what if there is a count bucket overflow? wrap-around or saturated increment?
-* do i even need to hash the hashcodes before using them as IBF bucket indices?
-
-* playing around with sim.c: your 2nd improvment seems to increase std dev by 
about 1/3
-
-* http://grothoff.org/christian/rrsize2012.pdf
- * "the expected maximum number of leading overlapping bits
-   between all n random node idenfiers in the network and a random key"
-  * is that the "greatest overlap common to all n peer identities and a random 
key"
-    or the "greatest overlap between a random key and one of the n peer 
identities"? (should be latter)
-
-
----------------------------------------------------------------
-
-* collisions are actually happening unexcpectedly often, am I doing something 
wrong?
-* if i don't hash the (random) hash again before using it for the bucket id, i 
get *far* more collisions. why??
-* performance is not that good, any suggestions?
- * using smaller hashes / faster hash function
-* can / should I use alloca?
-* measuring time: should i do it this way?
-
-* introduce parameter "exchange frequency" for consensus begin
-
-* idea for consensus api:
- * conclude callback gives list of consensuses, with peers in consensus and 
and an ID
-  * => not too much changed
- * the ID can be used to to get rid of the excluded elements, something like
-  * iterate_excluded_elements (ConsensusId id, ConsensusRemoveIterator 
iterator)
-
-
-merge of gnunet-consensus-start-peers:
-* mailed sree harsha about testbed questions, said code in 
gnunet-testbed-profiler is outdated anyway, wants to merge them himself.
-
-
-
-