Re: MAINTAINERS: macOS host support (was: MAINTAINERS: take edk2)

[Daniel P . Berrangé]

On Sat, Mar 12, 2022 at 02:51:21PM +0100, Christian Schoenebeck wrote:
> On Freitag, 11. März 2022 10:26:47 CET Daniel P. Berrangé wrote:
> > On Fri, Mar 11, 2022 at 10:13:24AM +0100, Christian Schoenebeck wrote:
> > > On Donnerstag, 10. März 2022 12:40:06 CET Philippe Mathieu-Daudé wrote:
> > > > +Stefan for overall project resources.
> > > > 
> > > > On 10/3/22 12:07, Daniel P. Berrangé wrote:
> > > > > On Thu, Mar 10, 2022 at 12:00:35PM +0100, Christian Schoenebeck wrote:
> > > > >> On Mittwoch, 9. März 2022 12:44:16 CET Daniel P. Berrangé wrote:
> > > > >>> On Wed, Mar 09, 2022 at 11:40:42AM +0100, Christian Schoenebeck 
> wrote:
> > > > >>>> On Mittwoch, 9. März 2022 11:05:02 CET Philippe Mathieu-Daudé 
> wrote:
> > > > >>>>> Not sure what you have in mind. I'm totally new to the
> > > > >>>>> macOS/Darwin
> > > > >>>>> world, and have no choice but to use it as primary workstation and
> > > > >>>>> for CI builds, so I can help with overall testing / maintenance.
> > > > >>>>> 
> > > > >>>>> Peter, since you take some macOS patches, would you like to
> > > > >>>>> maintain
> > > > >>>>> this officially? Since I doubt you want to take yet another
> > > > >>>>> responsibility, what about having a co-maintained section,
> > > > >>>>> including
> > > > >>>>> technical expertise from Akihiko / Joelle / Christian? (Cc'ed)
> > > > >>>>> 
> > > > >>>>> Regards,
> > > > >>>> 
> > > > >>>> Also CCing Cameron on this, just in case someone at Apple could
> > > > >>>> spend
> > > > >>>> some
> > > > >>>> slices on QEMU macOS patches in general as well.
> > > > >>>> 
> > > > >>>> As for my part: I try to help out more on the macOS front. As
> > > > >>>> there's
> > > > >>>> now
> > > > >>>> macOS host support for 9p I have to start QEMU testing on macOS
> > > > >>>> locally
> > > > >>>> anyway. Too bad that macOS CI tests on Github are no longer
> > > > >>>> available
> > > > >>>> BTW.
> > > > >>> 
> > > > >>> Note QEMU gets macOS CI coverage in GitLab. We use a clever trick by
> > > > >>> which we use 'cirrus-run' from the GitLab job to trigger a build in
> > > > >>> Cirrus CI's macOS builders, and pull the results back when its done.
> > > > >>> 
> > > > >>> Any contributor can get this working on their QEMU fork too, if they
> > > > >>> configure the needed Cirrus CI API token. See the docs in
> > > > >>> 
> > > > >>>     .gitlab-ci.d/cirrus/README.rst
> > > > >>> 
> > > > >>> This is enough for build + automated tests.
> > > > >> 
> > > > >> Does this mean that people no longer have to pull their credit card
> > > > >> just
> > > > >> for running CI tests on Gitlab?
> > > > > 
> > > > > Not really. The CC validation is something GitLab have had to force
> > > > > onto all new accounts due to cryptominer abuse of their free shared
> > > > > CI runners :-( If you have VMs somewhere you could theoretically
> > > > > spin up your own CI runners instead of using the shared runners and
> > > > > that could avoid the CC validation need.
> > > > 
> > > > Not that trivial, first you need to figure out the list of dependencies
> > > > GitLab images come with, then you realize you need 50GiB+ of available
> > > > storage a single pipeline (due to all the Docker images pulled / built)
> > > > and you also need a decent internet link otherwise various jobs timeout
> > > > randomly, then you have to wait 20h+ with a quad-core CPU / 16GiB RAM,
> > > 
> > > Considering that CI jobs currently take about 1 hour on Gitlab, which
> > > processor generation are you referring to that would take 20 hours?
> > 
> > You're not taking into account parallelism. The GitLab pipeline takes
> > 1 hour wallclock time, which is not the same as 1 hour CPU time. We
> > probably have 20+ jobs running in parallel on gitlab, as they get
> > farmed out to many machines. If you have only a single machine at your
> > disposal, then you'll have much less prallelism, so overall time can
> > be much longer.
> > 
> > > > and eventually you realize you lost 3 days of your life to not register
> > > > your CC which you'll be forced to give anyway.
> > > 
> > > It's an obstacle. And that keeps people away. Plus the trend seems to be
> > > that free CI services disappear one by one, so I am not so sure that
> > > giving your credit card once solves this issue for good.
> > 
> > The CC requirement there is primarily to act as an identity check
> > on accounts, so they have some mechanism to discourage and/or trace
> > abusive users. You can use it to purchase extra CI time, but they've
> > stated multiple times their intention to continue to grant free CI
> > time to open source projects and their contributors. They are actively
> > discussing their plans with a number of open source project contributors
> > including myself on behalf of QEMU, to better understand our needs. I
> > outlined my current understanding of their intentions here:
> > 
> >  https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg03962.html
> 
> Please send an announcement (in subject) and/or CC maintainers if there are 
> any news on this topic. This discussion went completely unseen on my end.
> 
> > > > Long term maintainers don't realize that because they had the luxury to
> > > > open their GitLab account soon enough and are now privileged.
> > > 
> > > Would it be possible to deploy all CI jobs via Cirrus-CI?
> > 
> > Not unless you want to wait 10 hours for the pipeline to finish. Cirrus
> > CI only lets you run 2 jobs at a time. It also doesn't have any integrated
> > container registry which we rely on for creatnig our build env.
> 
> For the vast majority of contributors that would be absolutely fine. What 
> matters is running tests for the various architectures. Average response time 
> on submitted patches is much longer than 10 hours. Still better than not
> running CI tests at all.

I don't think that's absolutely fine at all, nor a common view amongst
maintainers/contributors. People already complain that the 1 hour time
of our GitLab CI is too long for them to wait. Having a CI run take 10
hours would be horrendous. Run a CI pipeline on monday, it fails, on
tuesday fix the bug, run another CI pipeline, and got get the results
on wednesday. Your work is split over 3 days, instead of 2 hours today
with GitLab as it stands. That's assuming you got the fix right first
time too. A CI pipeline that takes 10 hours, is a pipeline that people
will not bother running most of the time.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|