cybop-developers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [cybop-developers] UnitTests


From: Christian Heller
Subject: Re: [cybop-developers] UnitTests
Date: Mon, 06 Mar 2017 15:39:58 +0100
User-agent: KMail/4.14.1 (Linux/3.16.0-4-amd64; KDE/4.14.2; x86_64; ; )

Hi Enrico,

> Referring the testing:
> I already had a deeper look into the implemented tests. I hope you don’t mind 
> if I write you, why I think that the way how it is implemented should be 
> improved.

no problem with constructive criticism.

For historical reasons, I wrote short test functions when I needed them,
without having a full testing framework in my mind 15 years ago.

> 1. Fast feedback on where is what broken (the traffic light system on unit 
> tests)

But this requires an IDE with GUI, doesn't it?
Or, do you mean a message on console: OK/GREEN or ERROR/RED?

> I have seen plenty of prints in the unit tests you wrote, which are printing 
> the result on the screen, this will not lead to an easy overview on where is 
> what broken. Instead it is necessary to know the tests and to read all the 
> output in detail. This is time consuming and probably frustrating. Rather 
> than using prints, tests should fail if an expected value does not equal with 
> the actual one. This should be improved, either by using asserts (i really 
> don’t recommend this, there is no good feedback possible) or another 
> framework which supports this easily

I understand your comment and know TDD, of course.
However, a new framework brings new dependencies and overhead.
I don't say: "no", but have to be convinced by the framework.
The source should not get polluted by test functions.
Ideally, they should be bundled in an own directory, e.g. src/test/

> 2. testing unit’s rather than integration
> Looks like that most of the tests are integration tests, they are using 
> several parts of the program, which will easily lead to issues when changing 
> the program. Rather than testing processes, unit’s should be tested.

Puhh, this is quite abstract.
CYBOI is not OOP, just structured-procedural programming.
Nearly every CYBOI function is encapsulated in its own source file/module.

I'm not sure how you want to create or use units,
but go ahead and give it a try and us an example.

> 3. Connected with the source
> I think it would be better to exclude the tests from the source. This way it 
> will not be contained in the binary. A second advantage of the separation 
> would be that it is not necessary to adjust the tester.c all the times tests 
> are added or removed. Using the current approach will create a lot of code 
> which needs to maintained.

O.k., I agree. If you find a good solution to do this, go ahead.

> 4. automatic execution
> CTest would allow to execute all defined tests and also create some dashboard 
> for code-coverage results and so on. As far as i understand the structure how 
> it is at the moment, can’t be easily integrated into CTest.

O.k.

> Proposal:
> 1. Extracting the tester-directory out of the src directory, to e.g. tests

I agree. Is it possible to call it in singular, i.e. "test/".
Please create this new directory in the CYBOP project's root,
i.e. parallel to "src/" and "examples/".
("examples/" will also be named in singular one day)

> 2. Putting the classes into a src similar folder structure under tests.

Not useful. CYBOI consists of a number of directories.

The following are not testable, as they contain just values:
- constant
- variable

The logger is global and probably does not need special testing:
- logger

The tests themselves will be moved away from "src/" in the future:
- tester

The applicator just delegates function calls with one function:
- applicator
Not really useful to test this, as it could be done in CYBOL/XML as well.

The controller is not really testable:
- controller
It contains the signal/event processing loop, IRQ mechanisms, threads.

The ONLY directory I consider "testable" is:
- executor
Here are the basic functions, arithmetic, comparison, deserialiser etc.

Addition:
In the "test/" directory, I have written some functions to understand
the functioning of the C programming language, e.g. pointers.
These files and functions are not related to CYBOI or unit tests.
Probably, they would have to be factored out into documentation then.

> 3. rewrite the tests to use some proper mechanism to fail when expected != 
> actual

If you'd like to do this work, then please go ahead!

> 4. integrate the tests to CTest

O.k. If this is possible, then do it.

> I am eager to know what you think about the proposal.

A student from IT 2011 had the same idea and copied the whole "src/"
structure 1:1. In the end, it was not really helpful or necessary and
cost me a lot of time to clean up and roll back a lot of stuff, in
order to keep overview.

My suggestion for you:
1 Focus on CMake first, until it works.
2 Afterwards, try to implement (only) one or two unit tests.

If all turns out to be usable and work fine, then you may stepwise
extend unit tests and thereby understand the CYBOI architecture.
Then we'll see further.

What do you think?

Christian





reply via email to

[Prev in Thread] Current Thread [Next in Thread]