bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case

[Pedro Alves]

On 6/30/20 5:40 AM, Jacob Bachmeyer wrote:
> Pedro Alves wrote:
>> On 6/26/20 11:37 PM, Jacob Bachmeyer wrote:
>>  
>>> Pedro Alves wrote:
>>>    
>>>> And again, people run the GDB testsuite in parallel mode, so your
>>>> assumption that the testsuite would stop immediately anyway doesn't
>>>> hold.  One of the parallel runtest invocations bombs
>>>> out, while the others keep running, so by the time you get
>>>> back the prompt, other testcases already executed, and you don't
>>>> see the ERROR in the current terminal anymore.
>>>>       
>>> This is a fundamental problem with slicing up the testsuite external to 
>>> DejaGnu and running multiple instances of runtest -- output that is 
>>> supposed to be "last at the end of the run" can be hidden away by other 
>>> ongoing subset runs.
>>>     
>>
>> That's like arguing against parallelization, and saying that everyone
>> should run the testsuite in serial mode, and be OK with it taking 2 hours
>> instead of 30 mins to complete a run...
>>
>> The parallel mode aggregates the summaries from the different runtest
>> invocations as a single summary at the end,
>>
>> https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=contrib/dg-extract-results.py;h=30aa68771d491451f72d6dbd18f6d5ac339451b5;hb=HEAD
>>
>> so I don't see what problem here is unsolvable.  Again, the issue is that Tcl
>> ERRORs aren't tallied up on the DejaGnu summary, so the aggregated summary
>> doesn't contain those either.
>>   
> 
> The current version of the fix for this issue (commit 
> 61dc0cafad8845b3c668940ed2e574bd503d410f on temporary branch PR41918) 
> generates separator lines containing at least 10 hyphens around each error 
> when repeating the errors after the summary.  The leading separator line 
> currently begins with "ERROR: " due to limitations of the internal 
> interfaces, but plans are to eventually fix that and collapse the separator 
> lines between adjacent error reports if multiple Tcl errors occur in a run.  
> Each error report begins with "ERROR: in testcase " and ends with a Tcl 
> backtrace and a line of at least 10 hyphens.  Is this something the GDB 
> results aggregation tool can be extended to handle?

Your guess is as good as mine, since I never touched that code.

Thanks,
Pedro Alves