Automatic tests

[Hynek Hanke]

On 30.9.2010 13:43, Andrei.Kholodnyi at gmail.com wrote:
> > all the available SSIP commands (as described in
> > SSIP documentation) and check their replies
> agree, however there is c api which need to be tested as well.

Yes.

>
> > A good example is the Python API tests in
> >  src/api/python/speechd/_tests.py
> does it make sense to move it under src/tests/api/python?

I don't see a reason for that. The script that runs tests
should run the appropriate tests from all submodules,
including the Python API.

> > A set of interactive tests (e.g. priority resolution)
> I think they could be automated as well,
> e.g. you can wait for particular notifications /cancel, begin etc/ and 
> analyze them.

Good idea!

I was just saying that even interactive tests are not so bad,
if they are easy to evaluate (clear yes/no decision etc.)

> This I didn't get, why do you think that most of the tests are 
> installation dependent?

Most anything that has to do with speech has to do with
TTS systems and audio backends, both of them are external
tools.

So what I would like to be able to do is to
     1) Generate tarball/package
     2) Install on several virtual machines with different configurations
     3) Run autotests script to evaluate

Thus I think a separate script is most versatile, while
make target or even make distcheck is too specific.

Best regards,
Hynek Hanke