Re: [PATCH 2/2] testsuite,pickles: Re-write test for pickles using `unittest.pk`

[Mohammad-Reza Nabipoor]

Hi, Jose.

On Tue, Dec 08, 2020 at 10:09:22AM +0100, Jose E. Marchesi wrote:
> 
> Other than that, I LOVE the approach :D
> 


I'm happy that you liked it :)


> 
> 2) What about XFAIL, XPASS and the like.  Right now the test harness
>    catches E_assert and assumes it is a fail, but we have to cover the
>    other possibilities too.  We need to find a way to convey that
>    information to the test harness.  I have a change for the Exception
>    struct in mind, to put the location information in its own field
>    instead of `msg'.  But for now we are ok with just FAIL and PASS.
> 

In `unittest.pk` (in next patch `pktest.pk`) pickle, every exception is
considered as failure (not only assert ones).
User should take care of him/her expected exceptions.

And I don't think we need XFAIL, XPASS.
Expected failure can be expressed using negating the condition:

    assert (do_fail () != OK, /*error message*/ "unexpected success");

Or if exceptions are involved:

    var A = [1,2,3];
    try
      {
        A[3];
        assert (0, "unreachable reached!");
      }
    catch if E_out_of_bounds
      {
        /* assert (1, "expected exception"); */
      }

or,

    try
      {
        raise_exception ();
        assert (0, "unreachable reached!");
      }
    catch (Exception ex)
      {
        if (ex == E_out_of_bounds || e == E_io)
          assert (1, "expected exception");
        else
          assert (0, "unexpected exception: " + exception_str (ex));
      }


I think, this approach is simple and general.
This also can be documented in the manual to guide users how to write tests.

Also adding location field to exceptions will make error messages more useful.


Regards,
Mohammad-Reza