Re: [RFC PATCH-for-5.2] gitlab-ci: Do not automatically run Avocado integration tests anymore

[Daniel P . Berrangé]

On Fri, Nov 27, 2020 at 07:29:31PM +0100, Thomas Huth wrote:
> On 27/11/2020 18.57, Philippe Mathieu-Daudé wrote:
> > On 11/27/20 6:47 PM, Thomas Huth wrote:
> >> On 27/11/2020 18.41, Philippe Mathieu-Daudé wrote:
> >>> We lately realized that the Avocado framework was not designed
> >>> to be regularly run on CI environments. Therefore, as of 5.2
> >>> we deprecate the gitlab-ci jobs using Avocado. To not disrupt
> >>> current users, it is possible to keep the current behavior by
> >>> setting the QEMU_CI_INTEGRATION_JOBS_PRE_5_2_RELEASE variable
> >>> (see [*]).
> >>> From now on, using these jobs (or adding new tests to them)
> >>> is strongly discouraged.
> >>>
> >>> Tests based on Avocado will be ported to new job schemes during
> >>> the next releases, with better documentation and templates.
> >>
> >> Why should we disable the jobs by default as long as there is no 
> >> replacement
> >> available yet?
> > 
> > Why keep it enabled if it is failing randomly
> 
> We can still disable single jobs if they are not stable, but that's no
> reason to disable all of them by default, is it?

Agreed, the jobs which are known to be broken or unreliable should
be unconditonally disabled in QEMU as a whole. This isn't specific
to gitlab config - the qemu build makefiles/mesonfiles should disable
the problem tests entirely, as we don't want developers wasting time
running them locally either if they're known to be broken/unreliable.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|