[Qemu-devel] Re: spice in kvm-autotest [was: Re: KVM call minutes for Apr 5]

[Alon Levy]

On Tue, Apr 05, 2011 at 03:03:00PM -0300, Lucas Meneghel Rodrigues wrote:
> On Tue, 2011-04-05 at 20:08 +0300, Alon Levy wrote:
> > On Tue, Apr 05, 2011 at 01:27:48PM -0300, Lucas Meneghel Rodrigues wrote:
> > > On Tue, 2011-04-05 at 19:01 +0300, Alon Levy wrote:
> >
> > > > So I was basically talking about the added requirement of creating a 
> > > > client
> > > > connection (one or more) to a single vm.
> > > 
> > > Yeah, I think now I've got the idea. We can certainly think of some
> > > strategy to coordinate test execution in vms in one host, and have
> > > another bare metal machine that opens clients to the vms on that host.
> > > 
> > > Autotest has infrastructure to coordinate tests among multiple bare
> > > metal machines, we call that infrastructure 'barriers'. It works
> > > basically through TCP socket communication, we have an autotest library
> > > for that, and an API that allows coordination among multiple machines.
> > > We use that for example, to coordinate cross host migration tests.
> > > 
> > > So, it might be tricky (specially setting up the environment on the
> > > machine that will open displays and handle all the client opening
> > > part[1]), but certainly doable IMHO.
> > > 
> > > [1] This is the part where the LDTP thing would come to place - we need
> > > infrastructure to support opening the display, starting the graphical
> > > clients and interacting with them in an automated fashion, LDTP and
> > > dogtail are the means to do that.
> > 
> > This might be required just for stuff like automating applications in the 
> > guest
> > that have no existing infrastructure, but I think we should try to avoid 
> > this
> > because it's notoriously difficult to create and maintain gui tests. WHQL 
> > doesn't
> > need this.
> 
> Agreed.
> 
> > > > Regarding the dogtail/LDTP issue, they are about specific tests run 
> > > > inside the
> > > > guest, and they are certainly something we would leverage. But like I 
> > > > mentioned
> > > > on the call, there is a whole suite of whql tests that are display 
> > > > specific,
> > > > and don't require anything new. In fact, a few months ago I added 
> > > > support for
> > > > autotest to run one of them, resizing to all the possible modes - so I 
> > > > know I
> > > > don't need dogtail for significant portions of our testing. (sorry, no 
> > > > git link
> > > > - I'll clean it up and post, it's been done 10 months ago so probably 
> > > > won't
> > > > cleanly apply :)
> > > 
> > > The thing about WHQL is that it has its own test suite coordination
> > > program (DTM), that has to run on a separate machine/VM. So they have
> > > all infrastructure in place there. If we need graphical clients being
> > > started and controlled on a linux bare metal machine, I am afraid we'll
> > > have to resort to those GUI test automation frameworks I mentioned. if
> > > our test is geared more towards windows clients, then they won't be
> > > needed, sure.
> > 
> > oh, but the added requirement is just launching the client. So - say you 
> > already
> > have a guest running the whql test suite. The only addition would be to also
> > have a spice client connected to that vm at the time the whql test is 
> > running. you
> > don't need to do anything in that client - don't send it mouse events or 
> > keyboard
> > events. Just let spice-server do what it does in this case, which would be 
> > sending
> > the spice protocol messages caused by the guest activity, which only 
> > happens when
> > a client is actually connected.
> 
> Ok, I didn't know that it's not necessary to interact with the graphical
> client.

I'm saying most of the tests, certainly the ones we will start with (i.e. 
WHQL), won't
require this. On the other hand, some tests will certainly need to inject 
events to the
client, to test keyboard / mouse channels (like the sticky key bug we have atm) 
and that
would require either scripting the client (still no dogtail/etc.) or real gui 
automation
if we have to.

> 
> > Another thing - the client doesn't have to run on a separate vm, or even on 
> > a separate
> > process. An additional vm seems like overkill, you are already building the 
> > qemu user
> > space from source or a specific tarball, one of the patches on my private 
> > tree adds
> > building spice from git, this includes the client and server. But we also 
> > have python
> > bindings for our newer spice-gtk client, so you could even import it from 
> > the autotest
> > python process. The whole barrier thing seems way overkill for this.
> 
> Fair enough. At some point, if people demand that other scenarios
> (remote clients running on remote machines) be tested, then we can
> re-visit the barrier thing.
>