[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Automatically checking regtests (was: Re: Minor releases?)

From: Graham Percival
Subject: Re: Automatically checking regtests (was: Re: Minor releases?)
Date: Mon, 17 Aug 2009 14:35:05 -0700
User-agent: Mutt/1.5.18 (2008-05-17)

On Mon, Aug 17, 2009 at 05:17:08PM +0200, Reinhold Kainhofer wrote:
> Am Montag, 17. August 2009 16:08:36 schrieb Michael Käppler:
> >
> > > (nobody checks the regression tests for each release, for example
> > > -- and that's trivially done with a web browser!)
> >
> > That reminds me of an idea I recently had: Wouldn't it be possible to
> > automatically generate a sort of "checksum" for each regression-test
> > output-file and compare it with the former releases?
> Isn't this exactly what we already have (make test-baseline to create the 
> reference version and then "make test" to compare the new output to the 
> reference output...). For the results see 
> The 2.13.3 results are at: 

Yes.  Right now there happen to be a lot, but I think there's
generally around a dozen examples.

> Graham was referring to the fact that nobody seem to bother about looking at 
> those automatically-created regression results before or after a release.

Yes.  All it takes is bookmarking the site, checking it whenever
there's a release, and reporting any broken examples.  However,
nobody is willing to commit to do this.  15 minutes whenever
there's a release, which happens at most once every two weeks.

- Graham

reply via email to

[Prev in Thread] Current Thread [Next in Thread]