Re: Item groups in bug tracker: "incorrect result" too broad, and masked by "regression"

[LachlanA]

John W. Eaton wrote
> On 01/11/2016 08:20 PM, LachlanA wrote:
>> because "incorrect result" also sounds like the most appropriate item
>> group
>> for things like unexpected behaviour of the GUI.
> 
> I don't expect 
> that it will be possible to get people to choose the correct thing just 
> because we have better words.  For example, I see that many people now 
> think "crash" means "Octave gave me an error" instead of "Octave exited 
> unexpectedly with a fatal signal".  But at least we could choose better 
> categories for sorting once the reports are submitted.
> 
>> I proposed splitting "incorrect result" into two: one (perhaps "wrong
>> answer") clearly for numerical errors and one (perhaps "wrong behaviour")
>> intended for errors in other settings.
> 
> But even if something is not a numerical function (GUI behavior, for 
> example) incorrect behavior is still an incorrect result, I think.
> 
>  > Michael Godfrey suggested that
>> "wrong answer" may still be too broad and suggested "fatal error".  I
>> personally think this sounds like either a crash of Octave or a  bug that
>> crashes the users code, rather than a grossly wrong answer.  Perhaps
>> "gross
>> numerical error" is the right term, but to me that suggests something
>> like
>> amplified roundoff errors (e.g., 1 / (a-b) for  similar a and b).
>>
>> Does anyone have suggestions for this, or how to specify a regression
>> without masking the other item-group values?  Does anyone have
>> suggestions
>> for how to prioritise debugging effort, given most bugs have the default
>> priority and severity?
> 
> Other than that, I have no clue what is best.  Sorry.  But I did just 
> notice that I recently added "Performance" to the list of Categories, 
> when maybe it should be an "Item Group" instead?  I don't know.  I'm not 
> even sure now what the difference between "Category" and "Item Group" is 
> supposed to be.  Oh, I see that Savannah has the following descriptions 
> on the page that is used to edit fields and field values:
> 
> Before choosing new names for things, or adding more names, maybe it 
> would be best to define what it is you want to be able to categorize?
> 
> jwe

Thanks for the reply, jwe.

It is natural that people think "my code crashed, so I'll select crash". 
That particular confusion could be avoided by renaming it "Octave crash".

I agree that for GUI behavior, incorrect behavior is still an incorrect
result.  That is why I was proposing to replace the current generic term
"incorrect result" with the more specific terms "incorrect answer" and
"incorrect behaviour".

In this instance, what I wanted to categorise is those bugs that make Octave
fail (without notification) to produce "correct" numerical answers for
finite inputs, ignoring roundoff / truncation (which is covered by
"inaccurate result").  That explicitly ignores bugs in plotting, bugs that
throw errors and GUI behaviour.  It probably doesn't include handling of
NaN/Inf.  It could include Matlab compatibility if the same code fragment
runs under both but has different semantics, but that would complicate
things.

"Performance" is currently in both Category and Item Group.  As you suggest,
it may not be necessary in Category.

Thanks for your time,
Lachlan



--
View this message in context: 
http://octave.1599824.n4.nabble.com/Item-groups-in-bug-tracker-incorrect-result-too-broad-and-masked-by-regression-tp4674355p4674368.html
Sent from the Octave - Maintainers mailing list archive at Nabble.com.