Re: [Monotone-devel] A couple questions...

[J Decker]

On Thu, Jun 3, 2010 at 2:56 PM, Timothy Brownawell <address@hidden> wrote:
> On 06/03/2010 04:21 PM, J Decker wrote:
>>
>> Why does monotone read all revisions and certs in the database on a sync?
>
> One fairly basic feature monotone has is that certs can be added to
> arbitrarily old revisions. What this means for netsync is that just
> synchronizing the revision graph isn't enough, you also need to check
> information from every node.
>

Okay well that does explain that :)

>
> I think we don't want to store that kind of per-peer information just on
> general principle.

Yes, I can see that becoming complex, which is why I mentioned saving
it only on the client sync side, which is likely to have fewer remotes
than the central server side (which could have many identifiers for
the same client if the client is behind many firewalls and travels a
lot to various hotels, etc)... but on the client side probably only a
few servers are generally used to sync... a central or 3, and maybe a
couple peers...

even though it is a peer-to-peer operation, there is a side that plays
like a client, and a side that hosts service.

>
> What would probably work is to store a skip-list over the revision graph,
> and synchronize on first the leaves (well, probably near-leaves, say
> anything not covered by a higher skip-node) at each level and then either
> prune/iterate or just synchronize on all unknown-state revisions. The skip
> nodes could have a hash over all certs that they skip over.
>

But then you'd need to mark 'I definatly changed this revision way
back in history' ... but then when is that status cleared, since I may
have to push that to 3 servers....

>> probably less than 1% of synchronizations are new pulls, which would
>> require reading all revisions in the branches of interest.  (in which
>> case revision C will be blank or null).
>
> Further, most pulls are going to have a few (or possibly no) new revisions
> on one side and no new revisions on the other. While it's fairly common that
> two people are commiting on the same branch at the same time, I think this
> is merely a significant minority of commits (and syncs).

true, since many branches in the overall sync are probably untouched.

>
>> I would think it would be a SIGNIFICANT reduction in sync time (which
>> isn't really that long for 30,000 revisions in assorted branches which
>> takes about 10 seconds to read on a netbook) to do a quick check and
>> just pull from the last known revision to the tip on both sides to
>> sync, instead of reading the entire revision structure.
>
> It's not quite that simple, but a graph-aware synchronization (with
> appropriate cert summarization) would definitely speed up figuring out what
> to send. It's on my to-do list somewhere, I promise. :)
>

Glad to hear that, sorry to hear that there are features that prevent
quick optimizations.

> --
> Timothy
>
> Free public monotone hosting: http://mtn-host.prjek.net
>