[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GNUstep testfarm reference
From: |
Richard Frith-Macdonald |
Subject: |
Re: GNUstep testfarm reference |
Date: |
Sat, 5 Mar 2011 13:00:30 +0000 |
On 5 Mar 2011, at 12:34, David Chisnall wrote:
> On 5 Mar 2011, at 09:22, Ivan Vučica wrote:
>
>> I'm not sure I understand -- would one need to test under Cocoa or under
>> GNUstep/OSX? If Cocoa testing is needed, I could try and contribute
>> something without setting up GS/OSX :-D
>
>
> Running the tests under Cocoa is the goal. In theory, the tests should
> always pass on OS X, because differences between OS X and GNUstep (unless
> explicitly testing GNUstep extensions) are bugs. If a test for some Cocoa
> functionality works under GNUstep, but fails under Cocoa, then the test is
> wrong and we should fix it (and then fix the GNUstep code that it was
> testing).
Yes, that's the theory ... but unfortunately we need to address things
carefully and slowly ...
Some failures under OSX are cases where we are testing for behavior which was
correct in the past (eg from OPENSTEP or earlier versions of OSX), but which
have been broken in recent OSX.
In these cases we might want to update the tests (and behavior of gnustep) to
reflect current OSX behavior ... but need to do it with care in case the change
breaks existing apps.
Some are cases where the test is wrong (or perhaps just too picky) and again,
we want to change the test and gnustep ... but again with care ... we need to
fix any code which depends on the existing behavior before we change the
behavior.
A few may be cases where OSX is definitely buggy/bad. In this case we have to
decide whether to follow the buggy behavior or simply continue to report the
bug on OSX.
So ... getting all the tests to pass on OSX (or be treated as hopes that Apple
will fix things) is the goal ... but simply changing the testcases to pass on
OSX without first dealing with the effect of those changs is not an option :-(