Re: [PATCH v7 22/22] gitlab-ci: Support macOS 12 via cirrus-run

[Daniel P . Berrangé]

On Wed, Mar 09, 2022 at 01:50:34PM +0100, Philippe Mathieu-Daudé wrote:
> On 9/3/22 13:33, Daniel P. Berrangé wrote:
> > On Wed, Mar 09, 2022 at 12:58:42PM +0100, Thomas Huth wrote:
> 
> > > Basically fine for me, but can we really run additional cirrus-ci jobs by
> > > default? IIRC the parallel execution of those were quite limited for the
> > > free tier, so did you look close that we don't run into additional 
> > > timeouts
> > > yet, due to delayed cirrus-ci jobs?
> > 
> > You can run 2 jobs in parallel in Cirrus. Beyond that they
> > get queued/serialized
> > 
> > We have a 1 hour job timeout.
> > 
> > We have to expect jobs will sometimes run slower than normal.
> > 
> > IOW if we have a job on Cirrus taking 30 minutes normally, we
> > expect it will sometimes take 45 minutes.
> > 
> > All this means that if we want Cirrus to be reliable and not
> > time out, we can really only have 2 jobs by default, unless
> > we can get the job execution time down to around 20 minutes
> > to allow for serialization.
> > 
> > We used to have terrible problems with cirrus timeouts when
> > we were running 4 jobs concurrently (2 on staging and 2 on
> > master). We addressed that in 9968de0a4a5470bd7b98dcd2fae5d5269908f16b
> > by disabling the jobs on master.
> > 
> > IOW, we really need to choose 1 macOS job and 1 FreeBSD job
> > and others need to be marked manual.
> 
> Not sure which job to choose yet. Per the first google hits we
> still want to cover Catalina first:
> https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/

My general gut feeling is usually to prioritize testing older versions
as they tend to be more widely used, and we want to avoid regresions
on stuff that has been around the longest. Compatibility problems with
new releases tend to get reported by users/maintainers and would not
be regressions, but rather enhancements to support the new platform.

But it is not really an easy decision, as clearly we would prefer to
test both old and new if we had the ability.

Having one as a nightly job is a reasonable compromise. 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|