Re: [PATCH] ui/cocoa: Support hardware cursor interface

[BALATON Zoltan]

On Sun, 30 Oct 2022, Akihiko Odaki wrote:
> On 2022/10/30 14:20, Elliot Nunn wrote:
> > Akihiko,
> >
> > Thanks very much for reviewing my patch.
> >
> > I think that you were right to use the sprite-within-a-window approach,
> > and avoid warping the OS cursor. I tried to compensate for the error
> > that cursor warping causes in the subsequent mouse event, but there is
> > still some error getting through that makes the cursor feel "janky".
> >
> > But in absolute pointing mode, will it be possible to remove the guest's
> > code path from visual updates of the cursor? I find that under Mac OS 9,
> > this provides better responsiveness. I can think of two methods:
> >
> > 1. In absolute pointing mode, re-enable Cocoa's cursor and let the host
> > OS move it according to user input.
> >
> > 2. Keep the cursor sprite, but move it according to Cocoa's mouse
> > movement events instead of dpy_mouse_set events.
> >
> > I prefer option 2. What do you think?
>
> My patch has been only tested with recent Linux, but it certainly should be 
> ensured that it works well for old systems when upstreaming.
>
> First I'd like to know what display device you use. It looks like 
> dpy_mouse_set is used only by ati-vga, vhost-user-gpu, virtio-gpu, and 
> vmware.
>
> Also, can you give reasoning while 2 is preferred? 1 would allow to exploit 
> the hardware's feature for cursor composition, resulting in smoother 
> experience and a bit less power consumption. But there may be complications 
> it can cause so I have not decided which one is the better yet.

Maybe cc-ing Gerd as the UI maintainer is a good idea in case he can add 
some insight too.

I'm not sure about this and may be wrong but I have a theory that the 
problems with mouse tracking are caused by not taking mouse ponter 
acceleration into account correctly. I did not check it and can't prove it 
but I think the guest and host cursor get out of sync because QEMU does 
not know about pointer acceleration methods and parameters which are 
different for each OS (and also user settable) so it's hard to reproduce 
it in QEMU so it just assumes linear movement and tracks guest cursor 
assuming that. Then it has to correct it when mouse is moved faster and 
the pointer ends up at a different place than expected. Is this a 
plausible theory? If so then we may need to take pointer acceleration into 
account or move the guest pointer based on host values or maybe somehow 
change guest pointer acceleration values to match the host's but that 
seems to be quite difficult because of the number of different guests. But 
it you're writing a guest driver this may be an option too at least for 
that guest.

Maybe it would help if there was a clearer picture of how all this works. 
We have the mouse connected to the host that sends relative move events. 
Then this is forwarded to the guest which moves the pointer based on accel 
setting of its own while the host moves its pointer with a different accel 
setting. Did I get that right? Then we either need to sync these pointer 
accel settings or pick one and use that for both. To let the host draw the 
pointer we should change the guest accel settings to match the host's or 
somehow change it for the QEMU window on the host to match the guest's. 
Neither seems to be easy and probably needs something in the guest to do 
it so I'm lost at this point and it's just a vague idea that could be all 
wrong but maybe thinking on this path could lead somewhere.

Regards,
BALATON Zoltan