[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU
From: |
Avi Kivity |
Subject: |
Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU |
Date: |
Thu, 29 Dec 2011 18:46:04 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111115 Thunderbird/8.0 |
On 12/29/2011 06:26 PM, Anthony Liguori wrote:
> On 12/28/2011 11:21 AM, Avi Kivity wrote:
>> On 12/28/2011 06:42 PM, Anthony Liguori wrote:
>>>> In fact using linux as a guest negates that. First of all, which
>>>> linux
>>>> version? if it's fixed, you'll eventually miss functionality and
>>>> need to
>>>> migrate. If it keeps changing, so does your test, and it will keep
>>>> breaking.
>>>
>>>
>>> The kernel is a git submodule so it's a very specific version. Yes,
>>> we may need to bump the version down the road and obviously, if we
>>> have to change any tests in the process, we can.
>>
>> Having a full linux source as part of the build process detracts
>> somewhat from the advantages here.
>
> Except that binaries can be made available. The initramfs is
> currently 512k while the kernel is about 5MB.
>
> OTOH, a Fedora install runs in the multiple GB. Having
> kernels/initramfs for the dozen target types that QEMU supports is
> reasonable. Having Fedora installs for all of them probably isn't for
> most people.
Then use DSL for the ping test.
>
>>> But there's certain things that I still consider to be unit testing
>>> (like basic networking tests) that I don't want to have to write with
>>> qtest. I'm not up for writing a TCP/IP stack in Python...
>>
>> A ping test is not a unit test.
>>
>> The ping test is already covered by kvm-autotest; just set up a config
>> that runs just that; after the initial run you'll have a guest installed
>> so it'll be quick. If we have a DSL guest it'll even be very quick.
>
> Achieving this with kvm-autotest is not easy. I doubt that DSL has an
> automated install mechanism and the step file based guest installation
> is fincky at best.
>
> When we had the kvm-autotest day, none of the folks that tried to set
> it up who had never set it up before had something working by the end
> of the day. And that was with lmr available on IRC to answer questions.
>
> There is a huge setup cost with kvm-autotest.
That needs to be fixed, not workarounded. I previously advised Lucas
not to spend too much time on this, since the audience is limited and it
would be hard to amortize the effort. Now I see that this was wrong.
>
>> To test the various NIC emulations, you don't need a full TCP stack,
>> just like you didn't need to write an NTP implementation for qtest's rtc
>> test. Instead you just poke values until it sends out a packet. If you
>> want to test virtio-net with both direct and indirect buffers, you can
>> only do that with a unit test, you can't do it using a full linux guest
>> since it has its own ideas of when to use indirects and when to avoid
>> them (for example, it may choose to always avoid them).
>
> I am not advocating that we don't write proper unit tests...
Right, and qtest is dangerously close to making writing tests fun. But
qemu-test is a wierd cross between qtest and kvm-autotest.
>
>>> Acceptance testing is, "does Windows boot", "can I create three
>>> virtio-serial devices".
>>>
>>> Obviously, part of acceptance testing is, "does this set of functional
>>> tests pass".
>>
>> Seems like a very blurry line. Especially as the functional test is
>> weaker than either qtest and kvm-autotest. I now have to agree with the
>> others that it's duplicate functionality. Does it really matter whether
>> you're creating an image by compiling Linux and assembling and
>> initramfs, or by downloading Fedora.iso and installing it?
>
> Yes. One of them is built entirely from source and the other is doing
> something meant to be done on bare metal.
>
> How do you test ARM guests with kvm-autotest? There is no such thing
> as an ISO installer for ARM. Are we going to download random images
> from the internet?
Do exactly what you plan to do with qemu-test.
>
> Even if there is an ISO, installing a guest with TCG will take many
> hours. My local qemu-test repo can now fully bootstrap an initramfs
> for non-x86 targets (which involves building gcc and uClibc). Start
> to finish it takes about 30 minutes regardless of the target (since
> the build runs natively).
You can skip the install test and only do the boot test. Seed your disk
with images from qemu.org or by running the install. kvm-autotest does
this.
>
>> Would you add live migration testing to qemu-test? If yes, you're
>> duplicating some more. If not, you're not doing functional or coverage
>> tests for that functionality.
>
> I would add live migration schema testing, absolutely. I wouldn't add
> an acceptance test though.
>
> An acceptance test with live migration would be creating a guest,
> running it, and then seeing it live migration worked.
Then it's duplicate functionality. It's already been done, but the
setup procedure is hard, so let's do it again.
>
>>> I don't want to write a TCP/IP stack. We aren't just grabbing a
>>> random distro kernel. We're building one from scratch configured in a
>>> specific way.
>>
>> How does that help?
>
> Not sure I understand the question.
>
In what way is your specifically configured kernel's TCP stack better
than the random distro's kernel's?
--
error compiling committee.c: too many arguments to function
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, (continued)
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Avi Kivity, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/28
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Avi Kivity, 2011/12/28
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Dor Laor, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Avi Kivity, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Avi Kivity, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU,
Avi Kivity <=
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Avi Kivity, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Avi Kivity, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Anthony Liguori, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Avi Kivity, 2011/12/29
- Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Cleber Rosa, 2011/12/28
Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU, Christoph Hellwig, 2011/12/28