Re: MAINTAINERS: macOS host support (was: MAINTAINERS: take edk2)

[Christian Schoenebeck]

On Freitag, 11. März 2022 10:26:47 CET Daniel P. Berrangé wrote:
> On Fri, Mar 11, 2022 at 10:13:24AM +0100, Christian Schoenebeck wrote:
> > On Donnerstag, 10. März 2022 12:40:06 CET Philippe Mathieu-Daudé wrote:
> > > +Stefan for overall project resources.
> > > 
> > > On 10/3/22 12:07, Daniel P. Berrangé wrote:
> > > > On Thu, Mar 10, 2022 at 12:00:35PM +0100, Christian Schoenebeck wrote:
> > > >> On Mittwoch, 9. März 2022 12:44:16 CET Daniel P. Berrangé wrote:
> > > >>> On Wed, Mar 09, 2022 at 11:40:42AM +0100, Christian Schoenebeck 
wrote:
> > > >>>> On Mittwoch, 9. März 2022 11:05:02 CET Philippe Mathieu-Daudé 
wrote:
> > > >>>>> Not sure what you have in mind. I'm totally new to the
> > > >>>>> macOS/Darwin
> > > >>>>> world, and have no choice but to use it as primary workstation and
> > > >>>>> for CI builds, so I can help with overall testing / maintenance.
> > > >>>>> 
> > > >>>>> Peter, since you take some macOS patches, would you like to
> > > >>>>> maintain
> > > >>>>> this officially? Since I doubt you want to take yet another
> > > >>>>> responsibility, what about having a co-maintained section,
> > > >>>>> including
> > > >>>>> technical expertise from Akihiko / Joelle / Christian? (Cc'ed)
> > > >>>>> 
> > > >>>>> Regards,
> > > >>>> 
> > > >>>> Also CCing Cameron on this, just in case someone at Apple could
> > > >>>> spend
> > > >>>> some
> > > >>>> slices on QEMU macOS patches in general as well.
> > > >>>> 
> > > >>>> As for my part: I try to help out more on the macOS front. As
> > > >>>> there's
> > > >>>> now
> > > >>>> macOS host support for 9p I have to start QEMU testing on macOS
> > > >>>> locally
> > > >>>> anyway. Too bad that macOS CI tests on Github are no longer
> > > >>>> available
> > > >>>> BTW.
> > > >>> 
> > > >>> Note QEMU gets macOS CI coverage in GitLab. We use a clever trick by
> > > >>> which we use 'cirrus-run' from the GitLab job to trigger a build in
> > > >>> Cirrus CI's macOS builders, and pull the results back when its done.
> > > >>> 
> > > >>> Any contributor can get this working on their QEMU fork too, if they
> > > >>> configure the needed Cirrus CI API token. See the docs in
> > > >>> 
> > > >>>     .gitlab-ci.d/cirrus/README.rst
> > > >>> 
> > > >>> This is enough for build + automated tests.
> > > >> 
> > > >> Does this mean that people no longer have to pull their credit card
> > > >> just
> > > >> for running CI tests on Gitlab?
> > > > 
> > > > Not really. The CC validation is something GitLab have had to force
> > > > onto all new accounts due to cryptominer abuse of their free shared
> > > > CI runners :-( If you have VMs somewhere you could theoretically
> > > > spin up your own CI runners instead of using the shared runners and
> > > > that could avoid the CC validation need.
> > > 
> > > Not that trivial, first you need to figure out the list of dependencies
> > > GitLab images come with, then you realize you need 50GiB+ of available
> > > storage a single pipeline (due to all the Docker images pulled / built)
> > > and you also need a decent internet link otherwise various jobs timeout
> > > randomly, then you have to wait 20h+ with a quad-core CPU / 16GiB RAM,
> > 
> > Considering that CI jobs currently take about 1 hour on Gitlab, which
> > processor generation are you referring to that would take 20 hours?
> 
> You're not taking into account parallelism. The GitLab pipeline takes
> 1 hour wallclock time, which is not the same as 1 hour CPU time. We
> probably have 20+ jobs running in parallel on gitlab, as they get
> farmed out to many machines. If you have only a single machine at your
> disposal, then you'll have much less prallelism, so overall time can
> be much longer.
> 
> > > and eventually you realize you lost 3 days of your life to not register
> > > your CC which you'll be forced to give anyway.
> > 
> > It's an obstacle. And that keeps people away. Plus the trend seems to be
> > that free CI services disappear one by one, so I am not so sure that
> > giving your credit card once solves this issue for good.
> 
> The CC requirement there is primarily to act as an identity check
> on accounts, so they have some mechanism to discourage and/or trace
> abusive users. You can use it to purchase extra CI time, but they've
> stated multiple times their intention to continue to grant free CI
> time to open source projects and their contributors. They are actively
> discussing their plans with a number of open source project contributors
> including myself on behalf of QEMU, to better understand our needs. I
> outlined my current understanding of their intentions here:
> 
>  https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg03962.html

Please send an announcement (in subject) and/or CC maintainers if there are 
any news on this topic. This discussion went completely unseen on my end.

> > > Long term maintainers don't realize that because they had the luxury to
> > > open their GitLab account soon enough and are now privileged.
> > 
> > Would it be possible to deploy all CI jobs via Cirrus-CI?
> 
> Not unless you want to wait 10 hours for the pipeline to finish. Cirrus
> CI only lets you run 2 jobs at a time. It also doesn't have any integrated
> container registry which we rely on for creatnig our build env.

For the vast majority of contributors that would be absolutely fine. What 
matters is running tests for the various architectures. Average response time 
on submitted patches is much longer than 10 hours. Still better than not 
running CI tests at all.

Best regards,
Christian Schoenebeck