Re: libobjc2 release?

[Richard Frith-Macdonald]

On 14 Mar 2010, at 12:44, Nicola Pero wrote:

> But you *do* have a point about spurious/known failures in the test suite.  I 
> would recommend we aim for exactly 0 failures.  We should silence
> all spurious or known failures so that people can run the test suite with 
> confidence knowing that any failure can and must be reported. ;-)

Well of course zero failures is the aim (and I think the base tests are usually 
very close to zero failures on gnu/linux) ... but what do you do about bugs 
which nobody has got around to fixing?  It would help if people were interested 
in contributing and maintaining testcases of course, but I don't think anyone 
(and I include myself in this) really like writing testcases.

I've recently cleaned up four of the five failures in the base library 
testcases, but that still leaves me (on CentOS 5.4, Intel) with this:

--- Running tests in base ---

base/NSException/basic.m:
FAIL: working callStackSymbols ... if this has failed it is probably due to a 
lack of support for objective-c method names (local symbols) in the 
backtrace_symbols() function of your libc. If so, you might lobby your 
operating system provider for a fix.

    260 COMPLETED
      1 FAIL
   5232 PASS


I added the extra text about the probable reason for the failure, but this is a 
case I don't really know how to deal with ... there was old, working (at least 
on some systems) code for this function, but it depended on libbfd and David 
spotted that libbfd uses the GPL rather than the LGPL and we can't  link it 
with base without making base GPL rather than LGPL.  He wrote new code which 
works for most things, but not for stack symbols ... because the underlying 
function (backtrace_symbols()) does not work properly (at least, not on any 
version of libc I've used).
We presumably don't want to just ignore the fact that the method no longer 
works, but as far as I know there's no likelihood of a fix in the near future.


One possibility that occurs to me is that we could maintain a file listing 
'common' failures and change the testsuite script to filter failures through 
that file (eg with 'fgrep -v -f filename').
That might then give us output like:

--- Running tests in base ---

  260 COMPLETED
   5232 PASS
  1 IGNORED (low priority failure ... see log for details)