Re: [Denemo-devel] Master is badly broken

[Richard Shann]

On Tue, 2014-04-15 at 10:04 +0200, Éloi Rivard wrote:
> The refactoring part I pushed on master was needed to test musicxml
> files.
> 
> Ok then, I will do the file comparision first.
> 
> Do you see some other tests I could do ?

Well, some sort of sanity check would be good and easy to run once you
have all this infrastructure in place. I just committed a scheme script
to tests/integration-data. The idea would be that it would generate a
score using many of the denemo commands and save the result for testing
for regression.
It is very rough: it tries to save to a fixed filename at the end. This
needs to be replaced with saving to the value of a scheme variable which
could be set on the command line by invoking Denemo with -a "(define
filename xxxxx)" where xxxx is the required path.

It already shows problems with scripts - several go interactive, which
they shouldn't do, I commented these out. And when run with -n it
segfaults after generating a slew of critical errors. So other commands
in there are requiring gtk I guess.

I generated this score by setting the recording on in the scheme window
and then executing some commands. The interactive ones the needed
editing, of course. So once the various problems are ironed out I could
generate a new one with a more systematic exploration of the commands.

As you remarked, it will be good to generate a new .scm script each time
a new command is made - the .scm script would use the new command for
some realistic case and save the denemo score that results, and
optionally also export a LilyPond file for diff-ing. This will catch
regressions where a change in the behavior of some command results in
the new command failing - a common occurrence.

Richard


> 
> 
> 
> 2014-04-14 15:32 GMT+02:00 Richard Shann <address@hidden>:
>         On Mon, 2014-04-14 at 09:25 +0200, Éloi Rivard wrote:
>         > Ok .mxml test reading is on master now.
>         >
>         > I also pushed the "refactor" branch, with some refactoring I
>         need to
>         > import NWC files (that is on yet another branch). It mostly
>         renames
>         > some functions, and split some files into tinier files. Can
>         you please
>         > check that all seem correct to you ? I would like to merge
>         this before
>         > it becomes too difficult to merge.
>         
>         
>         I'm a little confused here - you have pushed some re-factoring
>         of the
>         code to master (I have not had a chance to do more than a few
>         minutes
>         testing on that - seems ok), but for the branch called
>         "refactor" I was
>         hoping we could have a comparison of the import from musicxml,
>         which
>         could catch some regression. (Comparison of reading in and
>         writing out
>         the examples would be good too). If this is some way off, then
>         I can try
>         switching to the refactor branch for some real-life tasks, but
>         I only
>         use a fraction of Denemo when doing that. I think you have
>         done the hard
>         bit - getting the tests running - it should be easy to do a
>         comparison
>         with previous output (a simple diff should be ok as a start,
>         false
>         alarms aren't a problem yet).
>         
>         Richard
>         
>         >
>         >
>         > Next step is comparing tested files (mxml, denemo) with some
>         expected
>         > files…
>         >
>         >
>         >
>         > 2014-04-08 11:25 GMT+02:00 Richard Shann
>         <address@hidden>:
>         >         On Tue, 2014-04-08 at 10:40 +0200, Éloi Rivard
>         wrote:
>         >         > I saw you added some .xml test files. For the
>         moment,
>         >         only .mxml files
>         >         > are loaded, I will edit the test so .xml are
>         loaded too.
>         >
>         >         It might be better to rename them from .xml to .mxml
>         - there
>         >         is some
>         >         funny container format that the sponsors of MusicXML
>         use, with
>         >         a
>         >         different suffix. Just .xml is rather ambiguous.
>         >
>         >         Richard
>         >
>         >
>         >
>         >
>         >         >
>         >         >
>         >         >
>         >         > 2014-04-04 18:34 GMT+02:00 Richard Shann
>         >         <address@hidden>:
>         >         >         On Fri, 2014-04-04 at 16:12 +0200, Éloi
>         Rivard
>         >         wrote:
>         >         >         > Test suite now tries to open, save and
>         re-open
>         >         every .denemo
>         >         >         file in
>         >         >         > "examples" and every .mxml files in
>         >         >         "tests/integration-data".
>         >         >
>         >         >
>         >         >         That's a good start, but I think it really
>         needs to
>         >         check the
>         >         >         saved .denemo file has not regressed. We
>         would need
>         >         to check
>         >         >         in a set of
>         >         >         expected .denemo files, which could be
>         done quite
>         >         neatly by
>         >         >         make check
>         >         >         doing a diff against the expected .denemo
>         file if it
>         >         exists
>         >         >         else copying
>         >         >         the .denemo file it has created into the
>         directory
>         >         of
>         >         >         expected .denemo
>         >         >         files. Once a "git add" has been done on
>         the
>         >         "expected"
>         >         >         directory then
>         >         >         make check will be doing a regression
>         test.
>         >         >         This is particularly needed while we have
>         no test
>         >         that any
>         >         >         scheme script
>         >         >         ended without throwing an exception from
>         Guile.
>         >         >         There are no scripts that deliberately
>         cause errors
>         >         now - I
>         >         >         updated the
>         >         >         one that did that - but many scripts
>         trigger an
>         >         error if the
>         >         >         user
>         >         >         responds strangely or if the user is using
>         the guile
>         >         command
>         >         >         line in
>         >         >         Denemo. Such a guile throw wouldn't happen
>         for
>         >         scripts we use
>         >         >         as tests
>         >         >         (such as mxml import) so we could put
>         something in
>         >         the current
>         >         >         guile
>         >         >         throw handler for script errors that make
>         check
>         >         could detect,
>         >         >         without
>         >         >         having to exit Denemo abnormally. (I'm not
>         sure what
>         >         that
>         >         >         would be in
>         >         >         the environment that travis uses - perhaps
>         a file
>         >         that is
>         >         >         written to if
>         >         >         the standard_handler() in view.c is
>         executed).
>         >         >
>         >         >         But even if with that I think we need to
>         check for
>         >         changes to
>         >         >         generated .denemo files, (the most common
>         problem
>         >         would be
>         >         >         that some
>         >         >         innocent-looking change would result in
>         the cursor
>         >         not being
>         >         >         in the same
>         >         >         position and then the score generated
>         would end up
>         >         completely
>         >         >         different).
>         >         >
>         >         >         Does that sound ok to you?
>         >         >
>         >         >         Richard
>         >         >
>         >         >
>         >         >         >
>         >         >         > If you want to do some more mxml checks,
>         you just
>         >         needs to
>         >         >         add some
>         >         >         > files in tests/integration-data
>         >         >         >
>         >         >         >
>         >         >         >
>         >         >         > 2014-04-03 0:29 GMT+02:00 Éloi Rivard
>         >         <address@hidden>:
>         >         >         >         Richard,
>         >         >         >
>         >         >         >         Does the "refactor" branch seem
>         ok to
>         >         you ?
>         >         >         >
>         >         >         >
>         >         >         >
>         >         >         >         2014-03-31 18:05 GMT+02:00
>         Richard Shann
>         >         >         >         <address@hidden>:
>         >         >         >
>         >         >         >                 Jeremiah - please don't
>         build from
>         >         the
>         >         >         latest commit
>         >         >         >
>         >         >
>         >
>         
> http://git.savannah.gnu.org/gitweb/?p=denemo.git;a=commit;h=af6ae4aaa753e77e5dada20f5328959cbe76971f
>         >         >         >
>         >         >         >                 as it breaks Denemo in
>         ways I have
>         >         yet to
>         >         >         fathom.
>         >         >         >                 Attached is a file
>         >         >         >                 which just crashed
>         trying to open
>         >         it.
>         >         >         >                 This file could be
>         corrupt
>         >         (because of the
>         >         >         crash) ...
>         >         >         >
>         >         >         >                 Richard
>         >         >         >
>         >         >         >
>         >         >         >
>         >         >         >
>         >         >         >         --
>         >         >         >         Éloi Rivard - address@hidden
>         >         >         >
>         >         >         >         « On perd plus à être indécis
>         qu'à se
>         >         tromper. »
>         >         >         >
>         >         >         >
>         >         >         >
>         >         >         >
>         >         >         > --
>         >         >         > Éloi Rivard - address@hidden
>         >         >         >
>         >         >         > « On perd plus à être indécis qu'à se
>         tromper. »
>         >         >         >
>         >         >
>         >         >
>         >         >
>         >         >
>         >         >
>         >         >
>         >         > --
>         >         > Éloi Rivard - address@hidden
>         >         >
>         >         > « On perd plus à être indécis qu'à se tromper. »
>         >         >
>         >
>         >
>         >
>         >
>         >
>         >
>         > --
>         > Éloi Rivard - address@hidden
>         >
>         > « On perd plus à être indécis qu'à se tromper. »
>         >
>         
>         
>         
> 
> 
> 
> -- 
> Éloi Rivard - address@hidden
>         
> « On perd plus à être indécis qu'à se tromper. »
>