Re: [Monotone-devel] [RFC] Monotone NETSYNC Hook Extension & Abstraction Layer

[Ralf S. Engelschall]

On Tue, Sep 25, 2007, Markus Schiltknecht wrote:

> Ralf S. Engelschall wrote:
>> - To achieve the rollback we now temporarily PER SESSION queue
>>   the database operations resulting from such a bundle of received
>>   {file,delta}*,revision,cert* chunks before actually writing it to the
>>   database.
>
> I'm not sure I fully understand what you are doing. From reading the patch,
> I think you are caching (in memory) one revision and it's certs until you
> can decide if it is acceptable, right? Do the certs of a revision always
> follow the revision in netsync?

According to our discovery, in regular NETSYNC data flow: yes. But one
could hack a Monotone client and force it to do it differently, so one
cannot rely on this.

> What do you do if a revision comes without the required certs? A malicious
> peer could upload a revision (without offending certs) first, and then
> upload the cert in a later netsync run.

If this happens we would first store the revisions, of course. And when
the certs are coming those are rejected. We nevertheless would still
keep the revisions. That's a little bit unclean, but doesn't hurt as
those revisions are then still not "connected" to the branches we try to
access control.

> AFAIK, for the policy branches, we were always assuming that netsync could
> very well store offending revisions and certs in the database. Checking
> against the policy would happen later on. Much like suspending branches
> works. OTOH there's the advantage of always having a 'clean' database. What
> do others think?

First, at the end I think we need both things: our NETSYNC access
control _AND_ the policy branches. Both tackle a similar problem,
just from different angles. Second, the "clean" database is what we
would like to have. But more important is that we can control who
stores revisions which are connected (via certs) to our official
branches. Because it is essential for us that on the official branches
only revisions exist from contributors who really acceeded the
contributor license agreement -- although we want to allow everyone
to share their ideas with us inside particular "branch areas" like
"openpkg.community.playground.<user>.*" or something liket this.

>> - In case any NETSYNC Lua hook denies the storage of some data we
>>   "rollback" at least the current _bundle_ by clearing the mentioned queue
>>   and dropping the NETSYNC session. This way we for instance never store a
>>   revision without its corresponding certs in case the certs cause a Lua
>>   hook to deny access to the database.
>
> Hm.. what if Alice got some revisions from Crazy Bob, then Alice and Carol
> want to sync. But Carol already has a lua hook denying revisions from Crazy
> Bob. Given your implementation, netsync would abort after the first
> revision from Crazy Bob. And Alice would probably be unable to push all of
> its (perfectly legal) revisions to Carol.
>
> For Alice to be able to sync again with Carol, she would have:
>
>  a) figure out why netsync aborted,
>  b) manually check and merge the policy (lua hooks) with Carol,
>  c) manually remove the revisions from Crazy Bob.
>
> It would probably be better not to abort netsync completely and just deny
> single revisions. I'm not sure, though.

Is also possible as I mentioned in my other reply. The hooks can return
"accept", "ignore", "rollback:<message>", "abort:<message>" for _each_
piece of the NETSYNC protocol and "ignore" does what you are requesting
here. We implemented this because we ourself are still unsure what the
best approach actually is. Hence the hooks allow all four possible
actions.

> While these hooks sometimes look to me like a short-sighted attempt to
> implement policy branches, I'd rather like to think of it as a building
> block for it. Although I'm not quite sure how feasible that is.

Yes, one should not confuse this with policy branches. It really is just
a step towards this. Actually, the whole NETSYNC Lua hook extensions and
the abstraction layer is fully generic and reusable. Our intention is
that the same can be used for notifications, for auto-mirroring, etc.
The use for access control is just one possible (although prominent)
consumer.

> BTW: your website seems to be down at the moment, at least I can't connect
> to www.engelschall.com.

Yes, I was interrupted in the middle of a software upgrade and so Apache
was down for 2 hours. Is already fixed.

                                       Ralf S. Engelschall
                                       address@hidden
                                       www.engelschall.com