Re: [Gnu-arch-users] GCC v. Arch address@hidden: Regressions on mainline]

[Tom Lord]

    > From: Andrew Suffield <address@hidden>

You idea goes in an interesting direction:

    > I'm thinking of a system along the lines of tinderbox, as used by
    > mozilla - there is a group of hosts which just sit there endlessly
    > building and running the test suite, against the most recent revision
    > which has not yet been tested on this target platform (in practice it
    > would be a little more intelligent than this, and allocate revisions
    > to hosts so that test runs would tend to cluster on individual
    > revisions, rather than being spread around).

    > When some revision reaches the threshold (for gcc this is "one clean
    > test run on any hosted target"), it's merged. If any test fails
    > (unexpectedly), then (a) immediate mail notifications are sent, and
    > (b) a search begins, running just that test (and therefore faster),
    > looking for a revision that either predates or postdates the
    > failure. If it finds one, it runs the full test suite; if that passes,
    > it's found an acceptable merge point. (This process runs in parallel
    > to the endless builds of recent changesets)

    > This system has an interesting property: the queue never grows. It is
    > held at an average of (commit rate) * (time for 1.5 build+test runs on
    > the slowest required target) revisions behind everything else. That's
    > going to be an hour or two for gcc - which is probably acceptable. By
    > simply adding more hardware the system will fill in details for more
    > revisions. Slower targets will fill in less revisions, but you still
    > get consistent monitoring (and if you're patient enough, you get told
    > which changeset caused the regression).

    > I've left out a whole host of details, 

Yes.  What I'm specifically not quite clear on is the branching model
you have in mind.    I gather it's:


        developer   via naked   dog pile   pqm +
        branches   -----------> branch    ----------> monomain
        (many)       pqm        (one)      testing

where "monomain" is a mainline that always lags a few hours behind
dog-pile mainline but in exchange, for some definition of "quality",
is of monotonically increasing quality.

That would certainly be an improvement for volunteer testsers, 
at least.

Of course, the next step is the even more radical idea of:


    dev
    branches ---> dog pile ---> monomain0 ---> monomain1 --> ...

where each monomainN is autoderived from monomain(N-1) according
to increasingly strong criterea (more tests run, sign-offs based
on external testing, etc.)

In the extreme, the chain could end with:

        --> release-candidate --> release

where the last arrow is crossed whenever the latest release
candidate has (at least) some set of named improvements (noted
parsably in patch logs).   A rule could be something like:

   Make a release if:

   1) the latest unreleased release-candidate is 6m1d old.

   2) the latest unreleased release-candidate has a pre-specified
      list of features

It could get weird for the developers to "context switch" between 
versions down the chain of monomain or might turn out to be easy --
I'm not sure.   It's interesting to wonder if release phasing could
be entirely eliminated this way and to ask how people would feel 
about a system that sometimes just pipes up and surprises everyone
by announcing that a new release is ready.


    > but I think this scheme can
    > work. Here's the last trick: it's completely orthogonal. "Proof of
    > concept" takes the form of the system using the dogpile branch as its
    > solitary feed; as developers observe that it works and works better,
    > they can individually elect to work on a different branch and have
    > that branch monitored for merging.

One thing I'm not clear about re GCC.

They _do_ do nightly testing.   They _could_ conceivably test more
frequently.   Is the reason they don't the hw cost?  not wanting
results any more frequently than that?   Difficulty in making the test
runs less slow?   Tradition?  What?

-t