bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case

[Rob Savoye]

On 6/26/20 3:53 AM, Pedro Alves wrote:

> you can say that I'm more discussing the default.  But really I'm more
> worried about making sure that the people maintaining the GDB
> testsuite (me included), and the DejaGnu people understand each
> other's needs better.

  As the author of the GDB testsuite, I understand quite well how it
needs to work, or at least used to. Testing GDB was the #1 requirement
that lead to creating DejaGnu. Both have evolved considerably over the
years, so working together on improving the current implementation is a
good idea.

  What I get out of this discussion, which is a good one, is how DejaGnu
should handle errors. Unknown is one case, but there others. Whether
something is a warning, error, UNRESOLVED, UNTESTED, UNSUPPORTED, etc...
is often up to the developer. Maybe what needs to be improved is better
tracking of issues in a test run.

> at ERROR: lines in .sum.  Those would be buggy, but then again, if
> those ERRORs were accounted for in the DejaGnu summary, they would be
> harder to miss.

  Right, one reason I started on database support with better analysis
tools. Diffing text files has limited functionality.

> That's why I think --keep-going should be the default.  When
> you're developing a new testcase, you'll run that testcase in 
> isolation a number of times.  It's very hard to miss any
> silly error then -- there's no need to optimize for this use
> case by default, IMO.

  I personally don't consider development or debugging done till I have
done a full testrun native and cross to catch the errors running a test
in isolation misses. Yes, I know this can take days, but I prefer that
to assuming a validation testrun is not going to have problems.

  I'll note that there is a $HOME/.dejagnurc, where you can set the
default to whatever you want. :-)

        - rob-