Re: [PATCH 0/5] QEMU Gating CI

[Cleber Rosa]

On Thu, Mar 19, 2020 at 05:33:01PM +0100, Markus Armbruster wrote:
> Peter Maydell <address@hidden> writes:
> 
> > On Tue, 17 Mar 2020 at 14:13, Cleber Rosa <address@hidden> wrote:
> >>
> >> On Tue, Mar 17, 2020 at 09:29:32AM +0000, Peter Maydell wrote:
> >> > Ah, I see. My assumption was that this was all stuff that you were
> >> > working on, so that I would then be able to test that it worked 
> >> > correctly,
> >> > not that I would need to do configuration of the gitlab.com setup.
> >
> >> So, I had to use temporary hardware resources to set this up (and set
> >> it up countless times TBH).  I had the understanding based on the list
> >> of machines you documented[1] that at least some of them would be used
> >> for the permanent setup.
> >
> > Well, some of them will be (eg the s390 box), but some of them
> > are my personal ones that can't be reused easily. I'd assumed
> > in any case that gitlab would have at least support for x86 hosts:
> > we are definitely not going to continue to use my desktop machine
> > for running CI builds! Also IIRC RedHat said they'd be able to
> > provide some machines for runners.

While GitLab let's you run x86 code for free with the "Linux Shared
Runners"[1], I don't think it would be suitable to what we're trying
to achieve.  It's limited to a single OS (CoreOS), single architecture
and really geared towards running containers.  BTW, if it isn't clear,
this is the approach being used today for the jobs defined on
".gitlab-ci.yml".

IMO we can leverage and still expand on the use of the "Linux Shared
Runners", but to really get a grasp oh how well this model can work
for QEMU, we'll need "Specific Runners", because we're validating
how/if we can depend on it for OS/architectures they don't support on
shared runners (and sometimes not even for the gitlab-runner agent).

> 
> Correct!  As discussed at the QEMU summit, we'll gladly chip in runners
> to test the stuff we care about, but to match the coverage of your
> private zoo of machines, others will have to chip in, too.
>

I'm sorry I missed the original discussions, and I'm even more sorry
if that led to any misunderstandings here.

> >> OK, I see it, now it makes more sense.  So we're "only" missing the
> >> setup for the machines we'll use for the more permanent setup.  Would
> >> you like to do a staged setup/migration using one or some of the
> >> machines you documented?  I'm 100% onboard to help with this, meaning
> >> that I can assist you with instructions, or do "pair setup" of the
> >> machines if needed.  I think a good part of the evaluation here comes
> >> down to how manageable/reproducible the setup is, so it'd make sense
> >> for one to be part of the setup itself.
> >
> > I think we should start by getting the gitlab setup working
> > for the basic "x86 configs" first. Then we can try adding
> > a runner for s390 (that one's logistically easiest because
> > it is a project machine, not one owned by me personally or
> > by Linaro) once the basic framework is working, and expand
> > from there.
> 
> Makes sense to me.
> 
> Next steps to get this off the ground:
> 
> * Red Hat provides runner(s) for x86 stuff we care about.
> 
> * If that doesn't cover 'basic "x86 configs" in your judgement, we
>   fill the gaps as described below under "Expand from there".
> 
> * Add an s390 runner using the project machine you mentioned.
> 
> * Expand from there: identify the remaining gaps, map them to people /
>   organizations interested in them, and solicit contributions from these
>   guys.
> 
> A note on contributions: we need both hardware and people.  By people I
> mean maintainers for the infrastructure, the tools and all the runners.
> Cleber & team are willing to serve for the infrastructure, the tools and
> the Red Hat runners.

Right, while we've tried to streamline the process of setting up the
machines, there will be plenty of changes to improve the automation.

More importantly, maintaining the machines is very important to the
super important goal of catching code regressions only, and not facing
other failures.  Mundane tasks such as making sure enough disk space
is always available can be completely change the perception of the
usefulness of a CI environment.  And for this maintenance, we need
help from people "owning" those machines.

> 
> Does this sound workable?
> 
> > But to a large degree I really don't want to have to get
> > into the details of how gitlab works or setting up runners
> > myself if I can avoid it. We're going through this migration
> > because I want to be able to hand off the CI stuff to other
> > people, not to retain control of it.
> 
> Understand.  We need contributions to gating CI, but the whole point of
> this exercise is to make people other than *you* contribute to our
> gating CI :)
> 
> Let me use this opportunity to say thank you for all your integration
> work!
> 
>

^ THIS.  I have to say that I'm still amazed as to how Peter has
managed to automate, integrate and run all those tests in such varied
environments for so long.  Major kudos!

- Cleber.

[1] https://docs.gitlab.com/ee/user/gitlab_com/#linux-shared-runners

signature.asc
Description: PGP signature