chicken-users
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Chicken-users] Unit testing


From: Kon Lovett
Subject: Re: [Chicken-users] Unit testing
Date: Sat, 18 Nov 2006 21:24:09 -0800

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 18, 2006, at 8:27 PM, Alejandro Forero Cuervo wrote:

<snip>

However, you want to separate this as two sets of tests.  If the
"cons" crashed (in the real world this "initialization" code will
probably be far more sofisticated than just a cons), we wouldn't want
the whole set of tests to fail, just those that depend on it.  So the
above should probably be rewritten as:

    (test-group

        (define x (cons 1 2))

        (test-assert (pair? x))
        (test-equal? (car x) 1)
        (test-equal? (cdr x) 2))

    (test-group

        (define y (list 1 2 3))

        (test-assert (pair? y))
        (test-equal? (length y) 3))

That means that the only top-level forms should be macros from the
test API (and perhaps the (use ...); if that fails it can be assumed
that all tests will fail).

I realize I didn't properly explain what I mean by this in my previous
mail.  I'm sorry.

Gotcha. Setup & Teardown. "test-infrastructure" does this in a similar manner, lexically scoped variables + a "side-effect" & "destructor" form. "check" has explicit mechanisms & "testeez" is lexically scoped; neither is "safe". (Current release of test- infrastructure is not completely protected from exceptions in setup & teardown but will be.)


I actually run my tests compiled by default as it is a more
stringent  environment.

Well, running them both compiled and then interpreted has got to be
more stringent. ;-)

Umm, in practice I've found that compiled is enough when I am waiting for the results. For an automated system no, not enough. (In my environment an egg build always tries to perform any "X/*-test.scm" found after a successful build of "X".)


Changing manner in which test results are captured to a "write"
rather than a "cons" would be the big change.

Exactly, this is probably how I'd implement the test software. :-)

I'd define a protocol that allows a test to be executed by a certain
process and its result to be passed back to the parent process.  The
individual test forms would just fork, run the test and communicate
back the results (or, if they take too long, the parent would just
kill their process and record failure).  test-group in my examples
above would also fork and use the same protocol (though this time it
would be able to report the results of multiple tests back to the
parent).  And then you've got the parent that compiles the files
defining tests and also forks a separate process that sets its UID to
a "nobody" user and executes them.

Yeah, an automated system might get away with have this overhead - each "assertion" executed in its' own OS process. I still think that a package/group/case/whatever assertion (called "expectations" in "test-infrastructure") collection is a fine enough grain for an OS process.

- ---

I sense you a looking for a something that allows disparate individuals to add arbitrary tests (in the "TestUnit" sense, please ask if that isn't clear) to any Chicken Extension. Great idea. The devil is in the details.

I am not advocating "test-infrastructure" as the basis. It just already does a lot of what people want (even Felix uses it sometimes due to the HTML report - which I do not like but at least it exists). Also, do not forget that a lot Scheme code uses "testeez" & "SchemeUnit" already. (The 2 SRFI testing frameworks I don't think have much penetration. And "test-infrastructure" has none - except for me!) I wonder if some sort of "test result" coercion might allow all these different unit test frameworks to somehow slot in to a larger scale acceptance test framework.

I think the community can come up w/ something that meets the purpose of standalone testing (see the "test-it" form in "test- infrastructure"), per programmer unit testing, & acceptance testing (your concerns). (Integration test should fold into this but is application oriented.)

- ---

History:

Well over a year ago I had a back & forth conversation w/ Felix about Scheme unit testing. At the time the "test-infrastructure" seemed the most elaborate & "Scheme-ish" of the frameworks. So I began extending & bullet-proofing it. I never liked the form in which testing concepts are expressed by it but it does express a lot of testing concepts, and a lot more now.

The original generated a "s***-load" of code, the current version keeps the original's signatures but w/ much lower overhead.


Alejo.
http://azul.freaks-unidos.net/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iEYEARECAAYFAkVf6nkACgkQJJNoeGe+5O6UnwCaAgrMatPCyH/OlJCmwv6lD/bz
Y1sAnRBuPGNuSQV2+bEJ5sF/7p7B+Ryy
=TqWQ
-----END PGP SIGNATURE-----




reply via email to

[Prev in Thread] Current Thread [Next in Thread]