|
From: | Stefan Weil |
Subject: | Re: [Qemu-devel][Patch] Suggestion for testing framework |
Date: | Sun, 15 Jun 2008 20:52:55 +0200 |
User-agent: | Mozilla-Thunderbird 2.0.0.14 (X11/20080509) |
Balazs Attila-Mihaly (Cd-MaN) schrieb:
Hello all It seems that there is agreement that some sort of automated testing is "a good thing" ;-). I'll have some free time in the next couple of days and plan on throwing something like this together on a spare box. I was thinking along the lines: - several qemu images (one with Debian, one with Windows XP - I can get a free student license for it, etc) - a script does a checkout of the trunk, checks if the version number is different from the last checkout (to avoid spamming the list :-)) - the script introduces the source in each VM, starts the VM and lets the different compilers available in the VM (like gcc 3.3, 3.4, mingw) compile the source - if the compile fails, it collects the error logs - if the compile succeeds, performance and functionality tests are run with the resulting binary - the is the most nebulous part for the moment for me - if I recall Fabrice said that compiling something inside a VM is a good performance test... - results are sumitted to the list - if you are ok with that, I wouldn't want to spam the list Please comment if you find the testing methodology good and what performance and functionality test should the process include... Best regards. Hi, using an existing framework for continuous integration tests will provide many of the features needed. Here is an example for an automated mail (which might be sent to the Qemu mailing list). I used CruiseControl, a Java based framework. The configuration and a helper script are appended to this mail and could be added to Qemu trunk, so everyone can run his/her own continuous integration. The current configuration is just a simple compile test for all targets. It can be extended to add more information to the mail, to inform those who changed the code about success or failure, to run a web interface, to cross compile, to run tests... Of course, running the continuous integration on a server with access from the Internet would be better than running it on my private machine, but even my private machine could send mails to the list when something goes wrong. I tried to run CI on a virtual server but it did not have enough resources to run Java... Regards Stefan Example of mail sent by CruiseControl: View results here -> http://localhost/cruisecontrol/buildresults/qemu trunk x86?log=log20080615200821Lbuild.7
|
Index: tests/cruisecontrol/config.xml =================================================================== --- tests/cruisecontrol/config.xml (Revision 0) +++ tests/cruisecontrol/config.xml (Revision 0) @@ -0,0 +1,71 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- +CruiseControl configuration file for continuous integration tests of Qemu. +--> + +<cruisecontrol> + <!-- fix next two lines for your installation --> + <property name="srcdir" value="/home/stefan/src/qemu-trunk/trunk" /> + <property name="mailto" value="address@hidden" /> + + <!-- Native compilation with default configuration. --> + <project name="qemu trunk x86" buildafterfailed="false"> + <listeners> + <currentbuildstatuslistener file="logs/${project.name}/status.txt"/> + </listeners> + <bootstrappers> + <svnbootstrapper localworkingcopy="${srcdir}" /> + </bootstrappers> + <modificationset quietperiod="0"> + <svn localworkingcopy="${srcdir}" uselocalrevision="true" /> + </modificationset> + <schedule interval="600"> + <exec + command="tests/cruisecontrol/cc-build.sh" + workingdir="${srcdir}" + errorstr="build failed" + /> + </schedule> + <publishers> + <artifactspublisher file="${srcdir}/bin/native/make.log" dest="artifacts/${project.name}" /> + <htmlemail + mailhost="localhost" + reportsuccess="always" + returnaddress="address@hidden" + returnname="QEMU Developers CruiseControl" + subjectprefix="[Qemu-devel][CruiseControl]" + skipusers="true" + buildresultsurl="http://localhost/cruisecontrol/buildresults/${project.name}" + logdir="logs/${project.name}" + > + <always address="${mailto}" /> + </htmlemail> + + </publishers> + </project> + + <!-- Native compilation (64 bit) with default configuration. --> +<!-- + <project name="qemu trunk amd64" buildafterfailed="false"> + </project> +--> + + <!-- Some tests. Might also be added to one of the compilation projects. --> +<!-- + <project name="qemu trunk tests" buildafterfailed="false"> + </project> +--> + + <!-- Cross compilation (Win32) with default configuration. --> +<!-- + <project name="qemu trunk win32" buildafterfailed="false"> + </project> +--> + + <!-- Cross compilation (Mips) with default configuration. --> + <!-- Not working because of missing tcg support for mips host. --> +<!-- + <project name="qemu trunk mips" buildafterfailed="false"> + </project> +--> +</cruisecontrol> Index: tests/cruisecontrol/cc-build.sh =================================================================== --- tests/cruisecontrol/cc-build.sh (Revision 0) +++ tests/cruisecontrol/cc-build.sh (Revision 0) @@ -0,0 +1,19 @@ +#!/bin/sh + +# $Id$ + +# Build script for Continuous Integration with Cruise Control. +# Used by CI for Qemu. + +export LANG=C + +bindir=bin/native + +mkdir -p $bindir +cd $bindir + +../../configure + +(make || echo build failed) 2>&1 | tee make.log + +# eof Eigenschaftsänderungen: tests/cruisecontrol/cc-build.sh ___________________________________________________________________ Name: svn:executable + *
[Prev in Thread] | Current Thread | [Next in Thread] |