Re: shortening the git test suite

[Ricardo Wurmus]

Chris Marusich <address@hidden> writes:

> Ricardo Wurmus <address@hidden> writes:
>
>> Hi Guix,
>>
>> git takes a very long time to build, because it has an extensive test
>> suite.  Most of the time is spent in running the SVN interoperability
>> tests, though, which are not really all that interesting for most uses
>> of git.
>>
>> The Makefile says this:
>>
>>   # Define NO_SVN_TESTS if you want to skip time-consuming SVN 
>> interoperability
>>   # tests.  These tests take up a significant amount of the total test time
>>   # but are not needed unless you plan to talk to SVN repos.
>>
>> What do you think about disabling the SVN tests in the git package?
>
> This sounds similar to the discussion we had earlier about treating
> tests as a special case:
>
> https://lists.gnu.org/archive/html/guix-devel/2018-04/msg00071.html
>
> I felt that the conclusion of that thread was basically that if someone
> is concerned about the build time, then they ought to be able to use
> substitutes to speed things up, and we should continue to run as many
> tests as possible in order to discover problems sooner.

What I’m worried about is availability of substitutes.  When we keep
changing packages on the “master” branch that have very expensive test
suites then we accept that people won’t have substitutes for a while.
The duration of that while depends on how quickly the build of this
package is started by our continuous integration software, and how long
it takes to complete the build.

Granted, disabling parts of the test suite in an attempt to shorten it
is really a technical fix to a social problem.

Personally, I think that the SVN tests are non-essential (after all,
we’re building Git here, we keep running the individual test suites of
Git and Subversion, and git-svn interop seems like a thing that only
upstream need to worry about), which is why I made this proposal.  But
as it seems that the people who responded to this message rather lean in
the other direction, let’s try to address the social problem instead:

How can we change our workflow to make sure that for packages with long
build/test times we can provide substitutes more quickly?  Currently,
our policy for pushing changes to “staging” and “core-updates” is based
on package counts.  I’m not suggesting that changes to Git be pushed to
“staging”.

What do you think of pushing some package updates only to feature
branches that follow a certain naming convention (e.g. “_update-foo” for
updating the “foo” package), which causes Cuirass to build them and
merge the branch into “master” (semi-)automatically when the build is
successful?

(Obviously, any kind of automation has to be thought out carefully, but
I’m sure we would be able to find a solution.)

What do you think?

--
Ricardo