Re: SwiftUI compatibility APIs in GNUstep's graphics stack (Was: Which ObjC2.0 features are missing in the latest GCC?)

[Ivan Vučica]

We're straying off of the main discussion again now, but I'll do one
more response :-)

On Wed, Nov 27, 2019 at 6:02 PM Max Chan <xcvista@me.com> wrote:
>
> I wonder how Apple implemented their version though…

I will intentionally not discuss this further, and I have
intentionally not dug very deep. I don't want to be overly exposed to
ideas beyond the APIs.

But from the little that I had unfortunately observed, and from some
stuff I wish I hadn't read, yes, yes, WindowServer process and who
knows what else is involved in a lot of rendering of things in
windows.

> I do have an observation: if I force VESA graphics on my Hackintosh (this 
> requires special boot flags, so it is easier to do on Hackintosh than a real 
> Mac,) the OS interfaces are rendered very slowly. However if I allow 
> accelerated graphics (default behavior,) it operates smoothly but from time 
> to time the GPU fan would spin up even though there is no GPU-intensive tasks.
>
>
>
> I think Apple went down the exclusive OpenGL/Metal route.

Consider something else.

Let's imagine an X11 window manager that does compositing. Let's
imagine it assumes that the underlying platform *always* has OpenGL
available, and even the simplest 'skins' with animations switched off
will be painted with OpenGL.

And if you switch to something that has no OpenGL available, you
switch to using Mesa's software rendering. (Apple has, during its
OpenGL era on OS X, maintained a pretty good software renderer.
Although slow, you could opt to use it if you needed something
unsupported by the GPU.)

How fast would *just* blitting pixels onto the screen be? I mean, it's
certainly repainting the whole scene. 3D games repaint the entire
scene from scratch rather than updating bits of it. I'd wonder, what's
the restricting factor if you repaint the entire scene (i.e. do a lot
of pixel copies)? Where do these already-painted 2D windows end up, in
this hypothetical X11 compositing window manager? And, at that, let's
expand our hypothetical window manager: let's have it perform
compositing of shadows, or add opacity using alpha values, so that
there's a reason why it's using a 'compositing' window manager. How
fast would that work without any 3D -- or 2D! -- acceleration?

(Are you sure what this spins up is the GPU fan?)

Anyway, I'd expect that there's a lot of CPU-bound pixel copies
heading from main system RAM to main system RAM, then -- when
doublebuffering happens -- from main system RAM to whatever the
permitted primitive interface is (directly to video memory? I don't
know how this works on x86, (S)VGA/VESA, or in OS X). I'd expect
there's no magic sauce at all enabled. So, I'd expect window
compositing being done through a software GL renderer (even if the
underlying windows are unchanging 2D textures drawn only through Core
Graphics) can significantly ruin the framerate.

But that's just uninformed guessing on my part, without actually
testing anything on a real Mac.

> That is, either get accelerated graphics working at a kernel level, or 
> tolerate slow CoreGraphics.

Well, "slow".

Pixel copies, once something's painted with Core Graphics, can be
slow, too. I recall QuartzGL being a thing that Apple gave up on -- so
even though I have no idea what happened in the Metal era, I'd say
that most of the OS X (and iOS) systems have generally used software
painting of graphics into layers. Root-layers and their children then
get composited onto a scene.

We don't need to paint everything with GL. We don't even need to
composite layers into a scene using CARenderer, as updating
rectangular bits of an X11/Win32
pixmap/bitmap/whatever-rectangular-collection-of-pixels-you-want as
necessary can indeed be faster.



...having now advocated for "-gui doesn't need to *depend* on GL",
I'll say this:

Compositing with GL is pretty fast on most systems, so when/if
CAAppKitBridge is functional, I'd really go wild with optional
animations. :-)