[Monotone-devel] netsync status

[graydon hoare]

hi,

as of this past evening, the new "netsync" networking code in monotone 
is more or less complete. it probably still has a number of small 
errors, which I'll certainly continue to pursue, but the basic operation 
seems to work.

to recap the new functionality for those who have not been following:

  - netsync works monotone-to-monotone; no depots or NNTP/SMTP servers
    are used as intermediaries. one client serves connections while
    another connects to the serving one, so in a sense there is always
    a (minor) client/server relationship. but it is mostly irrelevant.
    either side can be the server, it just means they open the listening
    socket.

  - netsync is bidirectional. you can operate it unidirectionally --
    pushing or pulling -- but that is simply "turning off one
    direction". the default meaning of a netsync is to bring your
    collection into sync with the collection your peer has.

  - netsync keeps *no* history or logs of your discussion with other
    peers; instead it calculates (on the fly) the set of certificates,
    manifests and files missing from each side of a synchronization
    and exchanges them.

  - netsync uses a tricky sort of hashed index tree to find missing
    pieces. to exchange bits of this index quickly, the protocol is
    actually plain binary, application-specific, and pipelined. thus
    sacrificing all the sacred cows of monotone's old networking system.

  - the only state which can (possibly) get corrupted is your own
    database's indices, but they are built over your own data so can
    always be regenerated with a simple, local command.

anyways, I'll put a netsync "server" up shortly for people to play 
around with, and add a section to the monotone docs about netsync. it'll 
probably be marked as "experimental new feature" for the 0.10 release,
but if people are happy with it I'm somewhat tempted to throw out the
old network code and migrate to netsync permanently.

if you have questions or comments please ask.

-graydon