Re: [Qemu-devel] [PATCH 2/3] qtest: enable qtest for most targets

[Blue Swirl]

On Tue, Apr 17, 2012 at 20:31, Peter Maydell <address@hidden> wrote:
> On 17 April 2012 21:24, Blue Swirl <address@hidden> wrote:
>> On Tue, Apr 17, 2012 at 07:33, Peter Maydell <address@hidden> wrote:
>>> Just testing a device shouldn't require running a particular
>>> board model either, of course.
>>
>> The goal is obviously to make comprehensive tests for all devices in
>> all boards. Also, if a device is only used by a certain board then
>> yes, that particular board model is required.
>
> Device testing should be done by instantiating the device in
> a test harness and prodding it according to whatever tests
> you're doing. Trying to use the board as an ad-hoc test
> harness means you can't test aspects of the device which
> the board doesn't use, and if the board changes then half your
> test cases probably break.

You have a point. The unused features may be uninteresting though if
no board uses them.

>
>>>> I think we should probably refactor the boards to do something more useful
>>>> if a kernel isn't specified verses just buggering out.
>>>
>>> I don't inherently object if you can define something that's genuinely
>>> more useful. I'm not sure "start the CPU and have it execute through
>>> zeroed out RAM so the user gets no indication they've forgotten something"
>>> really counts, though...
>>
>> The error could be downgraded to a warning.
>
> Well, it could. But we should make that decision based on whether it
> makes sense and has a use case for actual users of the board, not
> because we're trying to get away with not having a setup that lets
> us properly unit test devices.

I disagree. I see many benefits in qtest and very little downsides in
changes like these.

qtest is a tool to let developers test the changes they make to
devices, so breakages shouldn't be so common. This should improve the
development process in QEMU tremendously.

It's also a tool to find bugs and improve code quality.

I value the above much higher than how user experience is slightly
(and also arguably) downgraded, because now they are able to shoot
themselves to a foot more easily.

>
> -- PMM