[Gnu-arch-users] Re: [arch-users] advanced usage advice: the prism technique (fwd)

[Andrea Arcangeli]

On Fri, Sep 26, 2003 at 09:19:21AM +0200, Pau Aliagas wrote:
> -------8<-------8<-------8<-------8<-------8<-------8<-------8<-------
> What does "prism merge" mean, as in "working prism merges into feature
> b"?
> 
> It means that instead of committing directly to "working", I commit in a
> few steps:
> 1) make some changes to working
> 2) decide which of the pure versions those changes belong on.
>    Call my project-tree for this version $PURE_TREE and its version $PURE.
> 3) in my project-tree for working, run:
>         tla what-changed -o ,changes [--diffs]
> 4) mv ,changes $PURE_TREE
> 5) pushd $PURE_TREE
> 6) tla redo -p ,changes
> 7) tla commit
> 8) popd
> 9) tla sync-tree $PURE
> 10) tla commit
> -------8<-------8<-------8<-------8<-------8<-------8<-------8<-------
> 
> -so your pure versions would be:
>  linux--aa-vm--2.4
>  linux--aa-newgen-schduler--2.4
>  linux--aa-new-driver-model--2.4
>  linux--aa-new-drivers-model--2.4

yes. That's exactly what I'm already doing by hand with normal patches
so it's very clear how it works ;)

So it's getting clear the best way to go is to simply have a branch for
each old-style "patch" I want to maintain in my tree.

What prevented me to do this so far and the reason I didn't find it
intuitive, is that branches are horribly expensive in all other revision
control software. here instead the "tag" (the hook) is immediate.

And I believe I was wrong about asking the ability to change the
patches and to name internal patches to each branch. But changing patch
in the middle is not what I want, the very single reason I find
attractive to use a revision control system is to have the log of the
evolution of each of these branches "i.e. my current patches".

I think this way of working wouldn't be feasible even with bitkeeper,
since I believe it can't be as fast as arch to generate a branch, but
really I've never used it so I can't be sure. Regardless the arch way of
doing things seems optimal.

About the "virtual" tag idea, I think I can withdraw it too. That
could allow me to also avoid any duplication between non-arch
directories and arch directories, but ideally I wouldn't need the
non-arch directories this way.

So now the only thing I care for efficient operation is that I can cache
the unpacked tree in unpacked form (and I believe it's already the
case), and that every new tree I checkout with 'get' gets forked sharing
the inode with an _hardlink_.  And the 10:1 optimization you told me
about for the inodes smells like doing exactly this. Is this correct? I
definitely need to share the inodes and their pagecache across all
unpakced directories I'm working on. (of course as far as they're all in
the same filesystem, otherwise the hardlink isn't possible, but that's
easy, cp -al just gets it right)

> You'd probaby import the patches into a working dir, move it to the 
> adequate branch using the prism technique, and then star-merge back and 
> forth to the right (aa, linus, marcelo). So you'd keep all your hard work 
> sorted, but transparent to the rest. Instead of seeing 20 patches, they'd 
> see as many as you'd choose to, depending on how you moved them from 
> working dir to pure versions.

yes, I think it's quite clear how it works.

Assuming I'm the only one working with arch (i.e. at the very least the
short term) however I don't see why should I star-merge into linus and
marcelo branches, they're read-only from my point of view, right?

> You'd keep all your patches properly sorted in all your branches (aa and 
> pure versions). Your aa tree would always be up to date related to 
> marcelo and linus because you'd resync with them using star-merge or 
> replay, so the non-merged patches would be always ready to export.

Yep.

> > One way I could achieve the above w/o arch changes could be to branch
> > the tree at the previous patch before the buggy one. Then I can do a
> > get-patch of the buggy patch piped into 'patch', and apply it by hand
> > (or maybe with tla redo not sure exactly, I don't know what redo exactly
> > does). Then I can commit it, the commit is basically the 'overwrite'
> > operation. Then I would need to reply all other patches from the
> > previous branch. Is this correct? Is there a way to reply all patches
> > from the other branch, except the one that I applied by hand already
> > with the bugfix?
> 
> I'd make it easier. Goto to the working dir, correct the bug and
> prism-merge it to the proper branch. Then propagate the change to aa.  
> Again, it feels natural.

sounds cool, hope it'll be as cool in practice as it is while thinking
about it now ;)

> You should never try to correct a patch, replacing it with another
> because, hopefully, people would mirror you archive and that means that
> they'd only get he new patches. You should just correct it with a new 
> patch and be done with it. It's a good policy documenting all the changes.

Yes I see.

I feel I was thinking in the wrong dimension because it's not intuitive
that a branch is zerocost.

> There are methods to remove a patch (tla replay --reverse) that could be 
> useful to remove an unwanted patch, but I'm not sure on how to use it 
> properly.

I'll leave them for a later study.

> > > that my revision library is perpetually up-to-date.   It's updated on
> > > every commit -- there's one line in my hooks file that ensures that.
> > > Otherwise, the scripts above would be painfully slow.
> > 
> > I can't understand this, what's this revision library, and what does it
> > mean perpetually up-to-date? if the scripts above are painfully slow
> > without this, then I guess I need this too?
> 
> Revision libraries are untarred versions, optimized via hardlinks to
> unchanged files of previous revision, of projects up to a patch level. 
> They are highly optimized in space use, so ot's like convnient having them 
> instead of reapplying a large number of patches.
> 
> You usually have a hook (it's a shell script) that adds a library to your 
> favorite branch every time you commit.

I thought it was closer to a .so extension loaded dynamically than a
bash script applying a patch to a cache tree, I see now what it means ;)




Actually I've still something unsolved. Basically each branch will be on
top of the other, I mean I can't keep them separated, they must generate
a tree, they must not only be good for merging, so my idea is:

        marcelo |tag> feature1 |tag> feature2 |tag> feature3

then my tree is "feature3". Checking out feature3 arch will unpack the
marcelo "cached" tree and it will start applying all patchsets in
the branch feature1, then it will follow the next tag and it will apply
all patchsets in feature2.

Let's assume I need to add a feature4, and that I know this feature4
has a good chance to be merged into mainline in a reasonable amount of
time (i.e. a security bugfix). Assume this new feature4 somehow collides
with feature3.  I clearly want to develop it on top of 'marcelo', not on
top of feature3. This way Marcelo won't run in rejects when he will
reply the 'feature4' branch in his tree (note: he will only pick
feature4, not feature1/2/3). If I would develop feature4 on top of
feature3, he would run in rejects.

Note, I do these ordering decisions all the time based on the
probability of what can be merged faster. I often add new simple fixes
at the very top of the chain (i.e. nearest to the marcelo branch)

So how can I add 'feature4' in arch?

First I create it with a tag on marcelo:

        marcelo |tag> feature4

Now the problem is that I must "re-tag" feature1 to point to feature4,
not "marcelo". then the chain will follow. Then I will get merging
errors while arch applies feature3, and I will solve them.

I can of course imagine that I can write a script that will generate
feature2a on top of feature4, and that I will star-merge feature2 into
feature2a. Is that the best way to do it? Don't you think it would be
better to have a native 're-tag' command as suggested above?

Is there really a difference at the end between a "re-tag" or a loop of
re-creation of branches that also would screw the name of the branch?

Andrea - If you prefer relying on open source software, check these links:
            rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/
            http://www.cobite.com/cvsps/
            svn://svn.kernel.org/linux-2.[46]/trunk