Re: [RFC] QEMU as Xcode project on macOS

[Christian Schoenebeck]

On Mittwoch, 9. September 2020 15:30:09 CEST Peter Maydell wrote:
> On Wed, 9 Sep 2020 at 13:56, Christian Schoenebeck
> 
> <qemu_oss@crudebyte.com> wrote:
> > I've recently been thinking about how feasible a stripped down Xcode
> > project for QEMU would be, i.e. you just get the QEMU sources, click on
> > qemu.xcodeproj, Cmd + B, done. No extra installation, no configure,
> > nothing.
> How would this work? Can you have an XCode project that just
> says "to build this program we will run configure and make" ?

You can run shell scripts. There are two main mechanisms in Xcode for that, 
one is "Build phases" where you can define shell code being executed for an 
individual "target" (e.g. a lib/app being built), which shell to be used (e.g. 
sh, zsh, bash, python), the actual shell code to be executed, and optionally a 
list of output files (e.g. .c files) said to be generated by the shell code 
which then in turn are compiled by Xcode according to their file type:

https://www.mokacoding.com/blog/better-build-phase-scripts/

Limitation: the precise list of generated output files needs to be specified 
to the Xcode project if they need to be compiled. So you cannot let the shell 
script dynamically tell Xcode "here are the source files I generated: [...] 
compile them please".

For that reason there is a 2nd mechanism in Xcode called "Build Rules", which 
basically allow you defining custom source file types, so e.g. Xcode would run 
a shell script for every source file that ends with "*.foo", the shell script 
would then generate for each one of them e.g. a .c file, which then would in 
turn be compiled by Xcode with clang.

In this example somebody auto generates .c files from .css files (sorry, dusty 
screenshots, but it's still the same thing):

https://www.cocoanetics.com/2012/02/xcode-build-rules/

> We definitely don't want to have two parallel mechanisms for
> specifying how to build QEMU, because they'll just get out of sync.

Sure, the Xcode project file should be auto generated for that reason, 
otherwise it's DOA.

> > each individually on macOS. Or right, you could alternatively "just
> > install" them from Homebrew, MacPorts, Fink. But no matter which solution
> > you choose, it easily ends up in a mess (conflicts, misbehaviours) on
> > macOS to install libs and apps globally. And I think that's the problem
> > why there are currently relatively little contribution for QEMU coming
> > from devs on macOS. Because you don't want to install things globally on
> > a macOS system, it's simply not working well there as it does with Linux
> > distros.
> 
> My experience with homebrew has been pretty good overall.
> If there's a better way to handle OSX hosts that doesn't
> require us to carry around all our dependencies (which is
> impractical) that would be interesting to investigate.

It probably depends on how many and which FOSS projects you compiled on Mac.
I no longer install from Homebrew and co, nor do I "pip install" things for 
the same reasons. And AFAICS other devs on Mac made similar experiences.

Plus I am used to FOSS projects having more issues on Mac then on Linux. If 
it's an Xcode generated app that crashes, then Xcode automatically opens the 
precise crashed lib's source file in the editor, along with the stacktrace, 
you fix the bug, click on "Run" and that's it. Without Xcode that would be 
like 45 mins just for recompiling the relevant dependencies.

On Mittwoch, 9. September 2020 15:40:05 CEST Daniel P. Berrangé wrote:
> Ideally any xcode setup would just invoke whatever our standard
> build tools are IMHO, so it has zero possibility of introducing
> new build problems.

Then you would not win anything on Mac. The problematic on macOS is that Apple 
froze many standard FOSS tools that switched to GPL3. So by default you just 
have e.g. GNU make 3.81 (2006), Bash 3.2.57 (2007), ...  unless you start to 
manually install them (e.g. from Homebrew & Co). And being forced to use Meson 
on Mac with all its Python dependencies does not make it easier.

IMO the Xcode project file would ideally be auto generated from the existing 
build infrastructure (e.g. Meson) and the scripts that are still required to 
be run by Xcode should be able to work with the old versions of build tools 
installed on a standard macOS system.

On Mittwoch, 9. September 2020 15:41:02 CEST Thomas Huth wrote:
> On 09/09/2020 14.56, Christian Schoenebeck wrote:
> > I've recently been thinking about how feasible a stripped down Xcode
> > project for QEMU would be, i.e. you just get the QEMU sources, click on
> > qemu.xcodeproj, Cmd + B, done. No extra installation, no configure,
> > nothing.
> Meson seems to have some exporter for Xcode according to
> https://mesonbuild.com/IDE-integration.html ... maybe you can harness
> that feature somehow?

Good to know, thanks! I'll give it a try.

> > The question is, and I don't have the big picture of QEMU yet to judge
> > that, how much is auto generated for QEMU i.e. with custom scripts that
> > would probably destroy this plan? There are these trace calls that are
> > auto generated, is there more like the TCG part for instance?
> 
> Yes, I think we generate code in a couple of places, e.g. the code in
> target/s390x/ uses a "gen-features" helper to generate some code. So
> implementing a separate Xcode project that does not use the main build
> files does not sound very appealing.

It's probably not that relevant how many files are generated, but more how 
many scripts approximately are generating these. If you just need to install 
like two dozens of shell hooks into the Xcode project, then Ok, that's 
manageable, if it's rather like hundreds or even thousands of script hooks, 
that would make the whole thing probably unrealistic, unless Meson's allegedly 
built-in Xcode feature already does a very great job. ;-)

On Mittwoch, 9. September 2020 16:40:10 CEST Programmingkid wrote:
> I think the solution to this problem is to switch over the CMake
> (https://en.wikipedia.org/wiki/CMake). It is a build system that lets you
> specify how you want to build software. There are many targets available
> including 'make' and an Xcode project file.

Well, the project just switched to Meson. It's probably not likely that there 
will be another build system switch any time soon I guess. But who knows, 
maybe Meson already does a good job on Xcode, or if not, maybe Meson could be 
fixed. We'll see.

Best regards,
Christian Schoenebeck