Re: [Denemo-devel] Master is badly broken

[Éloi Rivard]

Do you think the testsuite is tough enough to merge the "refactor" branch now ?

2014-05-19 20:37 GMT+02:00 Richard Shann <address@hidden>:

On Mon, 2014-05-19 at 20:11 +0200, Éloi Rivard wrote:
> What do you think of the testsuite at the moment ? I though about
> something, tell me if it looks a good idea to you.
>
>
> I create a new test that will try to execute every not builtin scheme
> command.
>
>
> The test parses the action directory to find the scheme commands.
>
> For each command, it checks if an associated test file exists, lets
> say tests/fixtures/scheme/THECOMMAND.scm. If so it executes it.

Is that to say it executes the script starting with a blank score? Does
it then save the score after the script has executed and test against
THECOMMAND.denemo ?
this would sound like a good framework for testing.

>  If not it just executes "(d-THECOMMAND)(d-quit)".
>
>  This would be a weak test in that case,

It could be made quite a bit stronger by making the environment in which
(d-THECOMMAND) is executed a more typical environment, by installing a
couple of staffs and some chords, leaving the cursor on a chord. Many
more commands do useful things when the cursor is on something and when
other staffs are present than do something useful in a completely empty
score.

(d-AddAfter)
(d-A)
(d-CursorUp)
(d-CursorUp)
(d-AddNoteToChord)
(d-MoveCursorLeft)
(d-THECOMMAND)
(d-Save "filename= ....")
(d-Quit)

would generate a distinctive output file for many commands (it creates
two staffs, populates one and then executes THECOMMAND in that
situation).

>  but it could at least check that the command does not provoke a
> segfault.

>
>
> Then the test could be a bit tougher. For example, we could decide
> that if a scheme command return FALSE, it makes the test fail.

I'm not sure that Denemo commands return anything useful. But detecting
scheme exceptions would be good - we could exit in the trap handler if
noninteractive was set. We might have to fix one or two commands that
don't expect to be executed in the given environment - well, we could
just write a test for them.

>
> What do you think ?

I think this is excellent - it will require a rule to generate the set
of expected output files, rather than diff them (for the initial
creation of the expected output), and one to accept an altered set of
files (copying them to the expected ones) would be a time-saver too.
After a change of version in the Denemo file format all the expected
output files would change - you would make just this change, check a few
examples and then run the rule to overwrite all the old versions with
the new ones.

Richard

>
>
> 2014-04-15 21:10 GMT+02:00 Richard Shann <address@hidden>:
>         On Tue, 2014-04-15 at 19:13 +0100, Richard Shann wrote:
>         > As you remarked, it will be good to generate a new .scm
>         script each
>         > time
>         > a new command is made
>
>         This script could assume that a variable,
>         Denemo-output-filename say,
>         was defined which it should use via
>
>         (d-Save Denemo-output-filename)
>         (d-Quit)
>
>         at the end of the test. (I think I missed the (d-Quit) out of
>         the
>         current script ...)
>
>         Richard
>
>
>
>
>
> --
> Éloi Rivard - address@hidden
>
> « On perd plus à être indécis qu'à se tromper. »
>

--
Éloi Rivard - address@hidden
       
« On perd plus à être indécis qu'à se tromper. »