Re: [Qemu-devel] State of QEMU CI as we enter 4.0

[Alex Bennée]

Daniel P. Berrangé <address@hidden> writes:

> On Thu, Mar 14, 2019 at 03:57:06PM +0000, Alex Bennée wrote:
>> Docker Images
>> =============
>>
>> The addition of docker has unlocked the ability to build a lot more
>> tests as well as compile testing on a much wider range of distros. I
>> think there are two outstanding areas that need improvement
>>
>> Daniel has been looking at building and hosting the images somewhere.
>> This would be useful as it would stop us slamming the distros
>> repositories constantly rebuilding the same images and also help reduce
>> the time to test.
>
> My intent was/still is to make use of quay.io for hosting prebuilt
> images.
>
> As well as avoiding repeated builds for developers it means that
> developers can be gauranteed to actually be testing with the same
> content that the automated CI did. Currently everyone using the
> docker images potentially has slightly different environment as
> it depends on what packages were in the repos when they built
> the image locally. This is very bad for reproducability.
>
> Libvirt uses quay.io for hosting images already and I've been
> looking at creating a script to automate usage of it via their
> REST API. Once done the same script should be usable by QEMU
> too.
>
> The idea would be that we still have docker files in the
> tests/docker/dockerfiles directory, but they would only be used
> for an automated job which triggers builds on quay.io, or for the
> few people who need to make changes to the dockerfiles. The current
> make rules used by developers / CI systems for executing test builds
> would be changed to simply pull the pre-built image off quay.io
> instead of running a docker build again.

Could we just have a script that pulls the quay.io image and tags it as
the appropriate target and then we could do:

  make docker-image-debian-arm64-cross [REBUILD=1]

which would normally pull the quay.io image but if REBUILD=1 would force
a local rebuild?

>> The other area that needs some work is better supporting non-x86 hosts.
>> While Docker's multi-arch story is much better (docker run debian:stable
>> will DTRT on any main architecture) we get stumped by things like
>> Debian's uneven support of cross compilers. For 4.1 I'd like to
>> reorganise the dockerfiles subdirectory into multiarch and arch specific
>> directories so we approach this is a less ad-hoc way. It would also be
>> nice to have the ability to gracefully fallback to linux-user powered
>> images where the host architecture doesn't have what we need.

I suspect we'd never store linux-user powered images on quay.io as there
are niggly differences depending on the users binfmt_misc setup.

--
Alex Bennée