[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Tests, Documentation
Re: Tests, Documentation
Tue, 25 Nov 2003 19:13:18 +0100
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030312
graydon hoare wrote:
Sascha Brawer <address@hidden> writes:
First, please write more test cases. I agree that it is boring, dumb, and
taxing; testing slows down development. But it's also the most reliable
way to make sure that code really works. If anyone needs to change our
code in five years, they'll want to know whether their change has any
serious impact. Without a large test suite, this is close to impossible.
I agree that we should be writing more test cases. but I think of it
in terms of a technical rather than moral failure: our testing system
requires mental energy beyond what we are willing to exert, and
rewards you with a sense of grinding tedium rather than excitement and
success. I think this is a result of mostly technical issues:
- mauve should be arranged such that:
- it's fetched, configured and built as part of classpath
- it builds and runs with the JVM / compiler combination you
configured classpath with
- tests can be added inline with a source file, or "right next
door" in a similarly-named class or package.
only if you want mauve to have GNU Classpath specific features.
- tests are run with "make check" from classpath, using $CWD as
the package restriction when in a subdirectory
- it uses simple introspection conventions rather than scanning
text in a source tree and writing makefiles for itself
java.lang.reflect? I think that one's out of question for tests for java
1.0. But I agree that reflection could make writing some tests easier. I
was thinking about using reflection for generating tests for
getter/setter pairs, for example.
- it should provide sub-frameworks for easily testing:
- comparability, equality, hashing, toString, etc.
I'm dealing with equality, hashing & toString atm, and hope to have
something to discuss on mauve-discuss based on some tests I'm writing
- container protocols
- threading protocols
- property (bean) protocols
- event listening / observer protocols
- null-pointer sensitivity
Add 'input sensitivity' to that. Many methods should check their input
for correctness. I believe some of classpath's IO classes
(BufferedReader ?) have problems with kaffe's regression tests for that.
- it should provide (or at least provide a place to hook up)
loopback, in-memory versions of:
- "remote" web server
- window system
- database server
- it should provide generic facilities for doing:
- pseudorandom / antirandom test vectors
- all pairs / all n-tuples test reduction
test reduction? could you elaborate?
- noise-tolerant boundary / range checking
again, culd you elaborate some more on that?
- coverage and profiling analysis
yes! when I write a mauve test, I'd like to know what code paths I've
missed. That should be possible to automatize.
- establishing and releasing invariants across methods
- establishing and releasing test prerequisites ("fixtures")
these are not too much to ask; existing JUnit-based suites often
provide some or all these as extensions to the basic TestCase class,
and they make testing feel much more productive.
without enhancing the framework -- judging from the tests we've
accumulated so far -- I don't think the mauve suite is ever going to
offer you much assurance about whether a change has "serious impact".
Yep. And it is not so much fun to write tests in, if you have to do much
of the above by hand for every other test you write.
Second, please write understandable documentation for every single
method. Developers should be able to create free programs without
referring to external documents. These could disappear at any time.
I think javadoc is a very poor documentation system. it makes the same
mistake as the rest of the java libraries: that verbosity and
factoring into ever-smaller parts is a prima facie duty.
It's better than a lot of the rest, i.e. no docs at all. You can process
documentation in various formats, etc. Of course something CWEB-ish
would be a great thing, but I think JavaDoc goes a long way into that
direction without requiring that you learn TeX ;)
imo it should not be. it only works in your favour up to a point, then
it works against you. to understand a package or logical concept in
javadoc requires you to have dozens of browser tabs open, flipping
back and forth and assembling knowledge in pieces, like a jigsaw
puzzle. it's a very unpleasant way of reading.
That's only the case if a class is badly designed and has a ton of
methods, fields & constructors. Of course, the JDK API is full of such
things. ;) But that doesn't mean that our implementations need to be
I would much rather each class -- or each *package* -- had 1 javadoc
comment, maybe with some diagrams or the like, explaining its
operation in broad detail and pointing out salient features and how
they relate to one another. method-by-method operation is not
particularly useful to document, especially since it's free code and
you can always just read what the method does.
That's what the top level comment of the class is for. To explain the
general usage of the class. But you also need to document the methods,
as the behaviour may be different depending on different states an
instance can be in.
Example: A closed stream can not be read from. That's easy to see from
the code, usually. But try to see what some methods in GregorianCalender
are supposed to do just by reading the code. ;) I assume that
understanding the Java2D code, if you're not proficient in 2D graphics,
would not be trivial either.
You shouldn't have to understand how it works in order to use it. That's
why you have JavaDocs on methods. To describe the user what she needs to
know in order to use it ;)
Re: Tests, Documentation, Mark Wielaard, 2003/11/23
Re: Tests, Documentation, graydon hoare, 2003/11/25
- Tests, Documentation, Sascha Brawer, 2003/11/22
- Re: Tests, Documentation, Patrik Reali, 2003/11/23
- Re: Tests, Documentation, Mark Wielaard, 2003/11/23
- Re: Tests, Documentation, Tom Tromey, 2003/11/23
- Re: Tests, Documentation, Brian Jones, 2003/11/24
- Re: Tests, Documentation, Mark Wielaard, 2003/11/25
Re: Tests, Documentation, Tom Tromey, 2003/11/25
- Re: Tests, Documentation,
Dalibor Topic <=