[Gnu-arch-users] Re: Why PQMs count

[Stephen J. Turnbull]

>>>>> "Tom" == Tom Lord <address@hidden> writes:

    >> Could you unpack that a bit?

    Tom> Glad to.

    >> I see Mark make a strong but polite request that somebody
    >> rectify an apparent policy violation; I don't see what that has
    >> to do with commit rate vs. branch rate.

    Tom> Read literally, the policy _is_:

    Tom>        "Don't commit without testing."

    Tom> Reading Mark's message in isolation, the policy apparently
    Tom> (to the casual reader) _means_:

    Tom>        "Don't make a breakage-inducing commit."

OK, I see where you're going now, but you can state the problem under
the literal policy, can't you?  The real question is "must you test on
the HEAD that you are proposing to commit to, or is a test on your
freshly updated workspace good enough?", right?

    Tom> So, in one sense, it's all a big sham.  The policy is a joke.

Only if you take the "no breakage in HEAD" interpretation, rather than
the literal "must test before commit" one.

But suppose you take an Einsteinian perspective, one which says
"simultaneous" has no meaning.  Then, OK, when your patch hits the
repository can be ordered with respect to mine in the repository's
time-line, but who committed first?  Einstein says that question may
not have an objective answer, since your timeline and mine may run at
different rates, from each other and from the repository.

_I think this is a good allegory for the problem at hand._ And it
points to the "no-breakage" interpretation as untenable, but "must
test" is still viable AFAICS.  Operationally, when you decide to
commit, you must update your tree.  Then you must build and test.
Then according to policy you may commit, even if for relativistic
reasons a "later" patch gets committed first.

    Tom> "Two months".  Implying that there is a shorter period of
    Tom> time that is _not_ "far too long".

    Tom> Now do you see where the mainline is for these folks?

Yes.

    Tom> I.e., it's just "dog pile on the mainline" and fix what's
    Tom> noticed as broken later rather than "Form an orderly line and
    Tom> make your proven-acceptable mainline changes one at a time.
    Tom> No shoving and no smoking, please."

Sure, but there are social reasons why most projects have a dog pile
at least some of the time.  Is the PQM etc part of the answer for
those projects, or is it "just" a technical improvement in Arch that
allows continuous commits (since they're always pulls by the PQM
managing a branch that is by definition up-to-date)?

    Tom> It's a continuous integration branch: they all hack on it
    Tom> using the modularity of GCC to make it probable but not
    Tom> certain they won't break this blind-integration branch ---
    Tom> and the value of keeping the branch itself is that when they
    Tom> are wrong, and do break something, this is picked up on very
    Tom> quickly.

    Tom> It wouldn't have been right for Mark post to say "Oh,
    Tom> somebody broke mainline."  (And that's not what he did.)  It
    Tom> _was_ right for Mark to say "Somebody broke mainline and
    Tom> timed out on fixing it.  Geeze, it's been two months!"

In other words, the breakage had become deliberate some weeks ago.
This is a social issue, and I don't see any technical solution.  In
fact, this kind of behavior is eventually going to screw other
developers badly.  Sure, we have to accept a certain amount of
breakage for probabilistic, SCM non-atomicity, reasons, but the more
probabilistic breakage there is, the less acceptable deliberate
shirking becomes, AFAICS.

I really don't see how a PQM can help relax that; it's simply not
acceptable for one person to break the branch that everybody is
working on merely because he doesn't have a good fix yet.  On the
other hand the PQM cannot deal with "relativistic time contraction" on
the one hand, and the social issues on the other, to decide that the
already committed patch is lower priority and should be reverted in
favor of one more recently submitted to the queue.

    Tom> Any such program has good changes being produced for it,
    Tom> around the "world" (literally the world, in GCC's case)
    Tom> around the clock.

    Tom> The rate of change, viewed as an irreducible natural fact, is
    Tom> greater than any human maintainer can keep up with.

s/any human maintainer can/the project maintainer wants to/.  It seems
to me that the technologies you're talking about may often be
appropriate as an optimization of another feasible process, rather
than useful only when it is the only feasible process (which you are
arguing is the case for "projects bigger than GCC", I think).

    Tom> In some sense, ahem, you could say that Arch (sans a PQM) is
    Tom> outrageously _backwards_.  Absent a PQM, Arch, in its move
    Tom> towards "whole tree orientation", takes the "C" out of "CVS".
    Tom> You can't commit unless you are up-to-date.

Commits are impossible _to the distinguished branch_.  It's plausible
that real projects need some "mainline" that is the consensus branch
for the next release, but it's not proved that I can see.  You could,
for example, take the Apache or Python processes as tentative
implementations of a process where individuals decide what the next
release should look like in terms of features, and then the developers
pick one of those integration branches, freeze, and fix all the bugs
in it.  In that model, the blockage you talk about is no problem,
because all the candidate branches are always up to date as measured
by themselves.

I guess you're saying that that is exactly what GCC _cannot_ do, but
GCC is a very exceptional case.  A thorough-going whole-tree advocate
could add special features for it, but insist "that's not the way you
_should_ do things when you can avoid it."  And that would pass for
99% of all projects.

    Tom> Oh, sure, we took from CVS the idea that you don't have to
    Tom> "lock" a file before you modify it --- but we won't let you
    Tom> commit that file (absent a PQM) unless not only that file but
    Tom> you _entire_tree_ is up-to-date.  Geeze, even SVN doesn't
    Tom> impose _that_ limitation.

But "tla commit --files" is Wrong, because it leaves an automatable
consistency check up to the programmer.  Granted, whole-tree
orientation for the whole GCC project is an Emersonian hobgoblin of
elephantine proportions, but on the other hand an SCM that allows you
to change a macro definition in module.h and commit that without
testing or checking up-to-date-ness of module.c is missing something.
Isn't it?

Here's a wild-ass idea for you: just as with build systems, an SCM
should have a notion of dependency, and _that_ dependency forest might
be a "natural" notion of "whole tree" which should be up-to-date
before a commit.  This might or might not be the same as build
dependencies.  I can see arguments for stronger and weaker, and for
putting the dependencies under (partial) control of the maintainers.

And you might even want to add features allowing searching for build
products to inventory and tree-lint.  Then they could (optionally)
check that the sources build, the tests were run, whatever.
Implementation will be hairy, I think, but conceptually well within
what tla's framework is already capable of, no?

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;