Re: [Qemu-devel] State of QEMU CI as we enter 4.0

[Alex Bennée]

Fam Zheng <address@hidden> writes:

>> On Mar 15, 2019, at 02:22, Peter Maydell <address@hidden> wrote:
>>
>> On Thu, 14 Mar 2019 at 15:57, Alex Bennée <address@hidden> wrote:
>>> Testing in the Cloud
>>> ====================
>>>
>>> After BuildBot went out-of-service we have been relying heavily on Travis
>>> as our primary CI platform. This has been creaking somewhat under the
>>> strain and while we have a large test matrix its coverage is fairly
>>> Ubuntu/x86 centric. However in recent months we've expanded and we now
>>> have:
>>>
>>>  - Shippable, cross compilers - catches a lot of 32/64 bit isms
>>>  - Cirrus, FreeBSD and MacOS builds
>>>  - GitLab, Alternative x86/Debian - iotests
>>
>> Are any of these capable of replacing my ad-hoc collection
>> of build test systems for testing merges ? I would quite like
>> to be able to do that, because it would make it easier for
>> other people to take over the process of handling pull requests
>> when I'm away.
>>
>> I think the main requirements for that would be:
>> * covers full range of hosts[*]
>> * can be asked to do a test build of a merge before
>>   I push it to master
>> * reliably completes all builds within say 90 minutes
>>   of being asked to start
>>
>> [+] I currently test:
>> - windows crossbuilds
>> - S390, AArch32, AArch64, PPC64 Linux
>>   (SPARC currently disabled because of the migration-test flakiness)
>> - OSX
>> - FreeBSD, OpenBSD, NetBSD via the tests/vm setup
>> - various x86-64 configs: from-clean; debug; clang; TCI; no-tcg;
>>   linux-static (including 'make check-tcg’)
>
> I think the gitlab CI architecture is quite capable of doing what you
> want here. Some efforts will be needed to set up the gitlab-runners in
> each of above environments and I expect tweakings will be needed to
> get the automation smooth, but it is fairly straigtforward and
> managable:
>
> https://docs.gitlab.com/runner/

If only it was that simple. It seems they don't current have arm64
packaged, only armhf:

  https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/725

I had installed the gitlab-runner from the Debian repo but it was out
of date and didn't seem to work correctly.

There build system seems... interesting.. in that it requires qemu-arm
to build on any host. I think it is because they are bundling all the
various architecture runners in one package which makes it a bit hard to
follow. Also it's in Go and has *all* the go dependencies which seems to
be a bit of a horror show in the distro packaging department.

However conceptually if we can get over that hurdle it does look as
though it could be quite promising. It's just getting to that point
might require a diversion to get gitlab's multiarch support up to speed.

By the way GitLab offer their additional tiers for free for FLOSS
projects:

  
https://about.gitlab.com/2018/06/05/gitlab-ultimate-and-gold-free-for-education-and-open-source/

so perhaps I should make the application for qemu-project. The main
benefit would be upping the total number of CI minutes we get on the
shared runners.

Shippable also offer a BYON (Bring Your Own Node) solution which we
should be able to plug one of the Packet.net arm64 servers into but it
seems to requite a node license to use so I haven't been able to play
with it yet.

(CCed: Ed from packet.net who runs the Works on ARM program)

--
Alex Bennée