Re: GNUism in groff tests, was: pic anomalies

[Ingo Schwarze]

Hi John,

John Gardner wrote on Tue, Dec 31, 2019 at 01:13:19PM +1100:

> Wouldn't having a TAP <https://testanything.org/> harness be
> preferable to hand-spun shell-scripts?

Introducing a test framework is a solution in search of a problem.  :-/

There are important things to do in groff.  It is quite an old
codebase, and much of it could profit from code review and code
cleanup (what we just saw in pic.ypp is merely an example, i
suspect most of the code may be looking similar).

Also, writing more tests, in particular for important features,
would be worthwhile, and doing so while reviewing code, right before
or together with refactoring, would be an ideal time.

If people want to work on feature improvements, that would be quite
useful, too; i believe we even sport a roadmap on the website.

Compared to that, introducing a framework for almost non-existent
tests does not strike me as urgent, nor even as a particularly good
idea at all.  Besides, the current tests already (mostly) work and
can be extended.

According to my experience, the hardest problem with tests in
practice is to organize them in such a way that, when suites grow,
it remains clear what is tested where and you can quickly find the
relevant tests when wondering "Is this or that already tested?
Where and how should i add a new test after fixing this bug or
improving this feature, or should i rather modify or extend some
existing test?"  And this one can also be hard to answer in practice:
"This test just broke.  What is it supposed to test, and why does
it no longer work?"  I believe none of these questions allow short
and easy answers, but they are what one should focus on when
writing tests.

A framework helps with none of that.  But a framework can easily
be so heavy that it distracts from the actual job.  Certainly,
we don't need a framework without content.

> I was rather shocked to learn such a widely-used program as Groff
> had such minimal test coverage.

As Branden observed, unfortunately it's still not unusual in the
industry.

When groff developement was most active (around 2000-2006), having
unit tests was even less common, and unit tests rarely spring into
existence when development slows down.

So many nice to haves...

Yours,
  Ingo