Re: [PATCH v2 0/5] gitlab: improvements to handling of stable staging branches

[Michael Tokarev]

26.05.2023 13:19, Daniel P. Berrangé wrote:
> We just (re)discovered that our gitlab rules don't work nicely with
> pipelines running from stable staging branches. Every pipeline gets
> published with the 'latest' tag, whether its the main staging branch
> or one of the stable staging branches. If pipelines for multiple
> staging branches run concurrently they'll get very confused and end
> up using the wrong container content. eg a 8.0 stable job will get
> run with a container from the development branch, or vica-verca.
>
> With this series we dynamically change the tag so that the 'staging'
> branch still uses 'latest', but the stable 'staging-X.Y' branaches
> use a 'staging-X-Y' container tag.
>
> We also let the container tag be set explicitly via the new variable
>
>    QEMU_CI_CONTAINER_TAG
>
> to facilitate CI testing, the new variable
>
>    QEMU_CI_UPSTREAM
>
> can be set to the fork namespace, to allow contributors to run a
> pipeline as if their fork were upstream.

Daniel, can you describe in a bit more detail (or refer to some text
to read) about how this whole thing works, aka the "big picture"?

It smells like we're doing huge wasteful job here, but it might be
just because I don't understand how it works.

Can't we prepare all containers separately and independently of regular
qemu commits, and just use the prepared container images every time
we run tests?

Also, why can't several tests (from several different pipelines, maybe
run for different branches) use the same images (if we don't re-prepare
them on each and every test run)?

I understand different branches might have different requirements for
containers, like using older version of some OS, etc, - this is done
by naming the container images appropriately, like debian-latest (for
master) vs debian-bullseye (for stable-7.2) etc.

Still, preparing container images might be done separately from regular
commits, like when ci config changes or when new version of something
(fedora/redhat/etc) is released.

How does it all works, why we need to couple container creation to
regular commits and re-create containers on every test run?

Thank you!

/mjt