Re: [Monotone-devel] partial pull #2 - gaps instead of a single horizon

[Markus Schiltknecht]

Hi,

Nathaniel Smith wrote:
> I assumed that in the normal case, one would do a partial pull to
> fetch the last chunk of history, but then only append to that going
> forward -- i.e., partial pull databases still grow over time at
> exactly the same rate as other databases, they just start out smaller.
> That seems like what most users would want, anyway.

Yes, that's probably sufficient for most people. But how do you explain 
them that partial pull works only once? That they have to throw away 
what they currently have, just to fetch the newest, bleeding edge revision?

> Not that this is really relevant to the question of gaps, because
> a rolling history window is still a contiguous history window.  I
> might not be understanding what you mean.

I'm thinking of the result of a partial pull as a repository having a 
gap between the root and the horizon. In that sense, such a repository 
it's not a single contiguous history window, because it also has a root. 
You would have to _replace_ the root with the sentinel revision ids to 
get a repository to be a contiguous history window.

> Anyway, from reading this thread so far, I'm not at all convinced
> that gaps are useful, or anything other than heinously complex to
> implement and document.  Ordinary single-horizon partial pull is
> conceptually straightforward, because all of our code _already_ knows
> how to deal with the horizon -- right now the "horizon" is hard-coded
> to always be "the root of the graph", but you can handle a lot of
> cases by just going through and tweaking all the loops that stop when
> they see the magic [] revision, to instead stop when they see a
> revision in the magic db.get_horizon() list.  I have no similar
> intuition about how to implement gaps; it seems like they'd need a
> whole pile of new machinery everywhere.

I disagree and claim the opposite: gaps, represented as ordinary 
revisions, are easier to implement, as all the code already knows how to 
deal with them. ;-)

There is an important differences between the root and a sentinel: a 
sentinel has a valid revision id and it can be replaced with real 
revision data, while the root can not. That's why we should print 
something for the user, when we hit a sentinel, but not when hitting the 
root.

My reasoning was: if we are going to implement sentinels for gaps 
between a horizon and the root, why not do it right and add support for 
gaps of any range? It's only marginally more work, while being a much 
more general solution.

> As for their utility, the most compelling use case seems to be that
> they could be used for the "obliterate" case ("the judge said we can't
> distribute this code anymore").  Are gaps really the right thing,
> though?  I don't have a great intuition here, but it seems to me that
> in the cases where you do have to delete such code, it's often stuff
> that was around for a while undetected, but restricted to a single
> part of the codebase -- so you don't want to delete your entire
> history between versions 1.1 and 1.3, you want to delete just that one
> file across that range.  And gaps won't help with that anyway.

Agreed.

How about hot servers? Or multiple partial pull? Or later reduction of 
your local repository size?

Most people, me included, do not understand why they absolutely must 
have the complete history of their repositories on all of their working 
machines. Gaps give us the freedom to choose how much data we want to 
carry - not only for an initial pull, but at any point in time.

> Am I missing something?

Uhm.. probably not, but some code can't help... I've just checked in 
what I currently have. Have a look at n.v.m.gaps, it includes a test 
doing a successful checkout, log and a log --diffs on a partial 
repository. Providing meaningful output for the missing revision.

Regards

Markus