[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Critical issues

From: Carl Sorensen
Subject: Re: Critical issues
Date: Thu, 13 May 2010 13:23:59 -0600

On 5/13/10 1:11 PM, "Karl Hammar" <address@hidden> wrote:

> Graham:
>> On Thu, May 13, 2010 at 10:06:45AM +0200, Karl Hammar wrote:
>>> How do one do a regtest?
>> Regression check; by compiling stuff.
>>>   8.1 Introduction to regression tests
>>>   The regression tests are automatically compiled using special `make'
>>>   targets.  The output of the regression tests is also automatically
>>> So, what targets?
>> They might be "make baseline-test", followed by applying a patch,
>> followed by "make check", but I'm not certain.  It's explained
>> somewhere in Contributing 2.
> Ok, found something in "3.6.3 Testing LilyPond" (though nothing in the
> chapter on regression tests).
>    * Initial test:
>           make [-jX]
>           make test-baseline
>           make [-jX CPU_COUNT=X] check
>    * Edit/compile/test cycle:
>           _## edit source files, then..._
>           make clean                    _## only if needed (see below)_
>           make [-jX]                    _## only if needed (see below)_
>           make test-redo                _## redo files differing from
> baseline_
>           make [-jX CPU_COUNT=X] check  _## CPU_COUNT here?_
> Hmm, the make check seems redundant since test-redo already does it:
> $ find . -name GNUmakefile | xargs grep -A 10 test-redo
> ...
> ./GNUmakefile:test-redo:
> ./GNUmakefile-  for a in `cat $(RESULT_DIR)/changed.txt` ; do \
> ./GNUmakefile-          echo removing $$a* ; \
> ./GNUmakefile-          rm -f $$a* ;\
> ./GNUmakefile-  done
> ./GNUmakefile-  $(MAKE) check
> ...
> ./scripts/build/out/output-distance seems to be the workhorse of the
> regression tests. I cannot find any useful documentation of it with:
>  find . -type f | xargs grep output-distance
> except the source code itself.
> But if I already have a known good result from the code tracker,
> how do I compare it with the new result?

What do you mean by "if I already have a known good result from the code

make test-baseline

followed by 

make check

will do exactly what you want.  You should then have, as described in
Contributors' 3.6.3, a file


that shows the results of the regression tests.

If the only snippet listed in out/test-results/index.html is, then that means you haven't broken any previous
regressions. is a snippet that has random content, so it will change
every time it's run.  That means that you will *always* get a difference
when you run the regression tests.

Ideally, in this case, you'd see changes in  And you'd
add a regression test for the current bug, so there would also be an entry
for the current bug.  Other than that, it would say that there is a large
number of snippets with no differences.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]