Re: make check recommended practice?

[David Bateman]

John W. Eaton wrote:
> On  3-Dec-2007, Moritz Borgmann wrote:
>
> | Now that I'm trying to chase down test failures for my Solaris build, 
> | I have a question about the "make check" command.
> | 
> | Whenever one compiles octave with only a subset of libraries (e.g., I 
> | don't include support for sparse matrices since we don't need it), 
> | then there is a large number of tests that are known already at 
> | configure time to fail. However, when looking at the output of make 
> | check or fntests.log, it is extremely tedious to look at every of the 
> | currently > 150 failures and determine if it's expected to fail or 
> | not.
> | 
> | How do people solve this problem? Is there some intelligence built-in 
> | that could keep the known-failing tests from running in the first 
> | place?
>
> We haven't done that because most of us doing development work on
> Octave have all the dependencies installed on the systems we use for
> working on Octave.  However, I'd consider a patch to avoid tests that
> can't work because of missing dependencies.  However, maybe instead of
> skipping the tests entirely, the test results summary should at least
> have a line with "N tests skipped because of missing build
> dependencies".  That way, people would at least be alerted to the fact
> that various things are not available because of the missing
> dependencies.
>
> jwe
>   
It wouldn't be too hard to disable the tests if the feature is missing.
To disable a test, there are examples in DLD-FUNCTIONS/regexp.cc where
there are some tests that are conditional on the presence of pcre.. 
They look something like

%!test
%! ## This test is expected to fail if PCRE is not installed
%! if (!isempty(findstr(octave_config_info ("DEFS"),"HAVE_PCRE")))
%!   ## TEST HERE
%! endif

You could do the same with any feature of Octave that is optional.
However, to add a flag for skipped tests would be harder. How would such
a test that might be skipped look like? Something like

%!testskipifabsent HAVE_PCRE
%! ## TEST HERE

and then the work to test if the feature is there would go in
scripts/testfun/test.m. Thinking about this, it probably won't be too
hard to do this, but what should the test name look like. I don't like
"testskipifabsent" ..

D.






-- 
David Bateman                                address@hidden
Motorola Labs - Paris                        +33 1 69 35 48 04 (Ph) 
Parc Les Algorithmes, Commune de St Aubin    +33 6 72 01 06 33 (Mob) 
91193 Gif-Sur-Yvette FRANCE                  +33 1 69 35 77 01 (Fax) 

The information contained in this communication has been classified as: 

[x] General Business Information 
[ ] Motorola Internal Use Only 
[ ] Motorola Confidential Proprietary