[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Tests, Documentation

From: Dalibor Topic
Subject: Re: Tests, Documentation
Date: Tue, 25 Nov 2003 19:13:18 +0100
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030312

Hi Graydon,

graydon hoare wrote:
Sascha Brawer <address@hidden> writes:

First, please write more test cases. I agree that it is boring, dumb, and
taxing; testing slows down development. But it's also the most reliable
way to make sure that code really works. If anyone needs to change our
code in five years, they'll want to know whether their change has any
serious impact. Without a large test suite, this is close to impossible.

I agree that we should be writing more test cases. but I think of it
in terms of a technical rather than moral failure: our testing system
requires mental energy beyond what we are willing to exert, and
rewards you with a sense of grinding tedium rather than excitement and
success. I think this is a result of mostly technical issues:

  - mauve should be arranged such that:
    - it's fetched, configured and built as part of classpath


    - it builds and runs with the JVM / compiler combination you
      configured classpath with

oh yes!

    - tests can be added inline with a source file, or "right next
      door" in a similarly-named class or package.

only if you want mauve to have GNU Classpath specific features.

    - tests are run with "make check" from classpath, using $CWD as
      the package restriction when in a subdirectory


    - it uses simple introspection conventions rather than scanning
      text in a source tree and writing makefiles for itself

java.lang.reflect? I think that one's out of question for tests for java 1.0. But I agree that reflection could make writing some tests easier. I was thinking about using reflection for generating tests for getter/setter pairs, for example.

  - it should provide sub-frameworks for easily testing:
    - serializability


    - comparability, equality, hashing, toString, etc.

I'm dealing with equality, hashing & toString atm, and hope to have something to discuss on mauve-discuss based on some tests I'm writing for FieldPosition.

    - container protocols
    - threading protocols
    - property (bean) protocols
    - event listening / observer protocols
    - null-pointer sensitivity

Add 'input sensitivity' to that. Many methods should check their input for correctness. I believe some of classpath's IO classes (BufferedReader ?) have problems with kaffe's regression tests for that.

  - it should provide (or at least provide a place to hook up)
    loopback, in-memory versions of:
    - "remote" web server
    - filesystem
    - window system
    - database server


  - it should provide generic facilities for doing:
    - pseudorandom / antirandom test vectors


    - all pairs / all n-tuples test reduction

test reduction? could you elaborate?

    - noise-tolerant boundary / range checking

again, culd you elaborate some more on that?

    - coverage and profiling analysis

yes! when I write a mauve test, I'd like to know what code paths I've missed. That should be possible to automatize.

    - establishing and releasing invariants across methods
    - establishing and releasing test prerequisites ("fixtures")

these are not too much to ask; existing JUnit-based suites often
provide some or all these as extensions to the basic TestCase class,
and they make testing feel much more productive.
without enhancing the framework -- judging from the tests we've
accumulated so far -- I don't think the mauve suite is ever going to
offer you much assurance about whether a change has "serious impact".

Yep. And it is not so much fun to write tests in, if you have to do much of the above by hand for every other test you write.

Second, please write understandable documentation for every single
method. Developers should be able to create free programs without
referring to external documents. These could disappear at any time.

I think javadoc is a very poor documentation system. it makes the same
mistake as the rest of the java libraries: that verbosity and
factoring into ever-smaller parts is a prima facie duty.

It's better than a lot of the rest, i.e. no docs at all. You can process documentation in various formats, etc. Of course something CWEB-ish would be a great thing, but I think JavaDoc goes a long way into that direction without requiring that you learn TeX ;)

imo it should not be. it only works in your favour up to a point, then
it works against you. to understand a package or logical concept in
javadoc requires you to have dozens of browser tabs open, flipping
back and forth and assembling knowledge in pieces, like a jigsaw
puzzle. it's a very unpleasant way of reading.

That's only the case if a class is badly designed and has a ton of methods, fields & constructors. Of course, the JDK API is full of such things. ;) But that doesn't mean that our implementations need to be equally messy.

I would much rather each class -- or each *package* -- had 1 javadoc
comment, maybe with some diagrams or the like, explaining its
operation in broad detail and pointing out salient features and how
they relate to one another. method-by-method operation is not
particularly useful to document, especially since it's free code and
you can always just read what the method does.

That's what the top level comment of the class is for. To explain the general usage of the class. But you also need to document the methods, as the behaviour may be different depending on different states an instance can be in.

Example: A closed stream can not be read from. That's easy to see from the code, usually. But try to see what some methods in GregorianCalender are supposed to do just by reading the code. ;) I assume that understanding the Java2D code, if you're not proficient in 2D graphics, would not be trivial either.

You shouldn't have to understand how it works in order to use it. That's why you have JavaDocs on methods. To describe the user what she needs to know in order to use it ;)

dalibor topic

reply via email to

[Prev in Thread] Current Thread [Next in Thread]