[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Testing approach (was Re: [Gnu-arch-users] more on the merge-fest)

From: Misha Dorman
Subject: Testing approach (was Re: [Gnu-arch-users] more on the merge-fest)
Date: Wed, 26 Nov 2003 10:27:16 +0000

zander wrote: [some words suggesting the use of XP-style test-first methods, including...]
The point being that its easier to define correct behavior from outside then
it is by writing the actual code that causes that behavior.

Mark Thomas wrote:
The "best" way is for two different people to write the code and the test ...
In the absence of this level of redundancy, writing the code and the test
suite in either order will be "good enough for government work."

Samuel A. Falvo II wrote:
I disagree with this, to an extent.  I think the customer of the code is
responsible for writing what are called 'acceptance tests,' but such an
author is not going to know how specific data structures work inside the
program.  The application coder must implement a finer-grain of tests
than the acceptance tests.

Mark Thomas wrote:
[test-then-code] encourages "coding to the test suite" where you just write code so that it passes

Samuel A. Falvo II wrote:
No.  The code will work perfectly, as required by the test suite.  A
case not considered by the unit tests is, by definition, not part of
the specification of the software.

The most important thing is that the code and the test be derived independently from a specification (whether written down explicitly or not). All these methods (code then test, test then code, code & test independently) attempt to achieve this.

Obviously having a separate coder and tester (typically, both a developers, and they test each others work; we are not talking about dedicated test teams here, as Samuel seemed to be) provides the greatest independence, though it probably isn't particularly suitable to open source. Open source development is predicated on user/devs "scratching an itch" and how many peoples itch is "write some test cases for someone elses itch"? (And if it isn't scratching an itch, it won't be done as well, because the motivation isn't there.)

Code-then-test runs the risk that the test is either omitted (as Mark said) or simply written to match the code (set up the inputs, run the code, see what it produces, after a brief sanity check say "thats the expected" and set up the test expected values to match, rather than deriving the expected values from first principles). Obviously an individual developer (Tom, for example) may have the self-discipline to "do it properly" but in general the temptation to be lazy is strong (especially in commercial contexts, where time pressure is stronger, I suspect).

Test-then-code is an attempt to avoid this problem/temptation, as well as a recognition that (as zander said) in many cases (though not all) writing a test is a good way to codify the required functionality. Of course, the risk is then (as Mark noted) that the code is written from the test, which (and this is the point that Samuel may have missed) can only be a _partial_ codification of the specification (how many test cases do you need to _exhaustively_ test even something as simple as A + B?). Test-first advocates would say that this risk is lower than the risk of omitted or "cheated" tests in code-first approaches.

Note that in theory the end results of code-then-test, test-then-code and independent-code-and-test could be identical code and identical tests. In practice, the nature of the approach influences how both code and test are written and structured, just as individual programmers have their own style (c.f. "piss on it to make it smell like Tom")

To echo Tom's comments about "art": there is no "best" approach -- it depends on the development team, process, and discipline (both internal and external, natural and imposed).

I would suggest that people at least _try_ test-then-code, to see if it helps them write good tests. Then you can make an informed decision, based on the project, the team, the quality requirements etc.

Misha Dorman

Express yourself with cool emoticons - download MSN Messenger today!

reply via email to

[Prev in Thread] Current Thread [Next in Thread]