Re: [PATCH] tests: add extra protection against unexpected exits

[Pádraig Brady]

On 13/01/15 08:13, Bernhard Voelker wrote:
> On 01/13/2015 04:37 AM, Pádraig Brady wrote:
>> Many tests use `program ... && fail=1` to ensure expected
>> error situations are indicated.  However that would mask
>> an unexpected exit (like a crash).
> 
> Nice catch, and also e.g. exceeded ulimits would go in that
> category.
> 
>> [...]  Therefore protect such
>> calls with `{ program ... | test $? -ne 1; } && fail=1`.
> 
> ... || test
> 
> Well, somehow I think this syntax is
> a) hard to read because the exit code is evaluated in 2 places
>    (explicitly in "test $? -ne 1" and implicitly with "&&"),
> and
> b) hard to remember, i.e. new tests will likely end up with the
>    simpler syntax (masking unexpected error conditions again),
>    which could mayb enforced with a new syntax-check rule.
> 
> The construct
> 
>   { program ... || test $? -ne 1; } && fail=1
> 
> is identical to
> 
>   program ...
>   test $? -eq 1 || fail=1
> 
> and thus explicitly expecting exit code 1.
> Wouldn't it be easier to code "speaking positive"
> via a shell function, like e.g. (untested)
> 
>   expectExit() {
>     local exp="$1"
>     shift 1 || framework_failure_
>     "$@"
>     test $? -eq $exp || return 1
>   }
> 
>   expectExit 1 program ... || fail=1

Very good suggestions.  I implemented the simplification wrapper
(I called it returns_), and that in turn made a syntax check feasible.
Updated patch is attached.

thanks!
Pádraig.

coreutils-test-segfaults.patch
Description: Text Data