Re: Remove references to test-output-distance.ly. (issue 549780044 by address@hidden)

[Han-Wen Nienhuys]

On Sun, Mar 29, 2020 at 11:45 PM Dan Eble <address@hidden> wrote:
>
> On Mar 29, 2020, at 17:39, Han-Wen Nienhuys <address@hidden> wrote:
> >> test-output-distance was removed on the grounds that the self-test
> >> serves the same purpose, but I don't see how it does.
> >
> > Could you elaborate? What failure scenario are you worried about?
>
> My question is, how does the self-test "verify that the regression tests 
> have, in fact, run"?  I don't see how it could do more than verify that 
> itself has run.

How does test-output-distance verify that the regression tests have run?

The verification doesn't come from the test file. It comes from the
fact that someone is looking at the test output.

The self-test gives us assurance that the test result index.html file
is making sense. You can verify that the tests were run by checking
that the index.html file is there.

For further context, I am interested in automating our test setup
further: I want to have CI that can post feedback ("test failed!") on
code reviews automatically. test-output-distance is a test that always
fails, which makes that more complicated.

-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen