[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Monotone-devel] Testresult

From: Judson Lester
Subject: Re: [Monotone-devel] Testresult
Date: Tue, 5 Jan 2010 15:25:26 -0800

On Sat, Dec 26, 2009 at 2:30 PM, Timothy Brownawell <address@hidden> wrote:
On 11/24/2009 1:48 PM, Judson Lester wrote:
Following the bisect thread brought some questions about testresult back
to mind.  This was one of the features of monotone that drew me to it,
back around 0.14, and I've never really gotten around to really
implementing testresult in my code projects.

Is there a best practice on using the mtn testresult command?

No. In fact I'm not sure that it's even used much, probably related to what you mentioned below. I think I know a better (more flexible) approach, but it would need new hooks to be added.

What would probably work would be a separate branch containing (only?) a file with the list of (test-id, pass/fail) pairs; revisions in this branch could be given an extra cert (because certs are indexable, and this is something you'd use for lookups) indicating which revision the contents refer to, and could maybe have an extra file with information on what version of the testsuite was used (for external testsuites).

Doing it this way would require a new "accept_update(old_revision, new_revision)" hook, and helper functions to allow that hook to look up particular revisions (use 'automate select') and then read the contents of those revisions (use 'automate get_file_of'). (We already have an 'mtn_automate' function that would allow these, except that it will E() on its reentrancy check if called this way.)

Does this seem like a reasonable approach?

Re-reading this, I started wondering if this might not be related to policy branches.  I am certainly going to reveal my novice-level understanding of deep version control, and possibly a complete misunderstanding of policy branches.  First of all, I know this was a hot topic in monotone about 3 years ago, but did they go anywhere?

It took me a little while to grasp the necessity of a second branch, although it's obvious on reflection.  In order to generate the testresults, you need to have the actual revision - you could only do that after the user issues a commit and before it's committed, and even then, that's a sketchy proposition.

So it makes sense, then, to put the results in a cert that can be added later - an actor can check out the revision, run tests and then sign certs on the revision.  Except for the complexity of the per-testresult, and the data bloat you alluded to.

What I started to think about then, was to commit the testresults in a clean format (maybe something parse_basic_io ready) into a branchless revision, and then sign a cert on the revision being tested with the rev_id of the testresult revision as the value. 

It seems like you could build a selector for the "associated branch" pretty easily, and be able to check it out and examine the contents of the testresult.

I think there'd be a hurdle with netsync, since I think (but don't know) if I'm syncing with a pattern, revisions that don't match the pattern would be excluded - so the associated revisions wouldn't make it over the wire. 

There's also questions about the DAG of the associated branch...

From there I begin to wonder: does that look at all like policy branches as they've been imagined?  All I've read is the Graydon proposal, that implies entirely separate branches - sort of filling in under (e.g.) with net, net.venge and net.venge.monotone with update policy. 


reply via email to

[Prev in Thread] Current Thread [Next in Thread]