[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding
Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding
Thu, 29 Mar 2007 21:55:23 -0500
Thunderbird 18.104.22.168 (X11/20070307)
Ramesh Dharan wrote:
Hi Anthony, sorry for the delayed response to your delayed response.
I actually haven't been working much on the remote display stuff in a while,
though it's something I'm hoping to get involved with again time permitting.
I'm cc'ing Dustin Byford and Johnson Liu, VMware engineers who handle a lot
of our remote display work these days.
Also, I'm wondering if it would make sense to set up a separate list so we
don't spam qemu-devel with this discussion, and also so that we can all
subscribe to it rather than having to be cc'ed and so forth. If you think
that's a good idea let me know. We can host the list @vmware.com, or put it
somewhere else (nongnu.org? or elsewhere?) if that would be preferred.
I'll send you a note and we can figure out the best place to host it but
I do think it's a good idea.
all the scenarios we care about)
Yes, right now, we do not address this.
So how do you handle the keyboard today? Do you just hardcode the reverse
mapping back to a pc105 keyboard?
We can take a keymap argument which can be used to map to various types
of keyboards (defaulting to a standard pc105 keyboard).
It would be nice to get something upstream so that all clients have the
ability to send scancode values. We have an extension in place which we can
just document. We use 16-bit "virtual scan code values" (they're 16-bit more
for convenience of implementation, i.e. directly mapped to a lookup table). I
believe the values we actually are equivalent/identical to PC scancodes.
Yeah, that would work out pretty well.
Meanwhile, in our newest version of Workstation (6.0) we added server-side
support for emulating different keyboard layouts when standard VNC clients
connect. Basically you can set a server config option to tell the server
"pretend that a de105 keyboard is attached" and it will do the reverse
translation of KeySym to Scancode accordingly. This is good enough for some
people, but the ideal situation is obviously to do the translation on the
client-side where you can actually detect the local keyboard and do the right
thing (which is what the VMware remote client does).
Yes, this would be much nicer.
There is a pseudo-encoding that provides alpha cursors with a 1-bit
alpha channel. An 8 bit alpha channel would be nice of course.
More than nice, it's pretty much required if you want to draw some modern
Linux guest cursors correctly. The cheap hack of treating < 255 alpha as
fully transparent doesn't really work that well, especially for things an
I-Bar cursor with shadow.
Right, I've seen this in action and it's not very pretty.
We do support the standard VNC pseudo-cursor encoding as of Workstation 5.5
and ESX 3.0, and if the guest sets an alpha cursor and the client doesn't
support our alpha cursor extension, we strip out the alpha channel and send
it using the standard VNC encoding. But it doesn't look that great.
This is something I do want to fix. This isn't just a server-side
issue, there is also a race in the protocol when a client uses
SetPixelFormat. A proper notification of when the server
format after a SetPixelFormat would solve this problem.
(4) no dynamic pixel depth/bpp change support
So yeah in the original protocol, depending on how you read it, it's just
illegal to ever send a SetPixelFormat after the original request. But at any
rate, client->server format requests aren't as interesting as server->client
requests. You only do the former if the client decides for some reason that
it wants a different pixel format, because e.g. it decides it doesn't have
enough bandwidth or something.
Server->client changes OTOH, can be guest driven, e.g. there are several mode
changes when booting a Windows guest. When a standard VNC client is
connected, we hide these from the client by doing pixel format conversion on
the server, which is expensive, and potentially wastes bandwidth on the wire
as well (e.g. when the guest is in 640x480x256 but the client wants 32bpp).
When a client which supports our "VMModeChange" extension is connected, we
just send down the mode change at the start of the next UpdateRequest.
Right now, we always do pixel conversion which is pretty annoying. It
would be very nice to be able to tell the client to deal with the
We do implement the standard VNC desktop size encoding (since otherwise we'd
have to just drop the connection every time the guest changed display size).
But getting back to your original point, since our client never sends mode
change requests upstream, so we haven't addressed the race that you describe.
I agree that if you ever want to drive these on the client you need to do
something about that.
Right, we're particularly prone to this race since our VNC server is
asynchronous. A SetServerPixelFormat message would solve this problem
since it would let a client know when it should start decoding pixels
In practice, most VNC servers do not support arbitrary pixel formats too
so it makes a lot of sense for server's to have this message available
(5) No notion of multiplexing displays on a single port.
Right. ATM, I don't really plan on addressing this.
So, the old Connectix Virtual Server product (which was in beta when
Microsoft snapped them up) had a super slick way of dealing with this, that
we actually joked about doing but never seriously thought anyone would do.
They rendered an entire "picker" UI using VNC primitives. The UI would show
live thumbnails of each VM, and then when you click on one of the tiles,
you'd get redirected to that console. You couldn't go back out to the picker,
but still, it was pretty neat.
I've since looked into doing the same thing, but the closest thing to any VNC
UI/widget rendering library that I've found is TclRFB (which, as the name
would imply, is written in Tcl).
Anyway, what you really want is a way to say "you've connected to a
multi-home server, here's a list of VMs available (with arbitrary or perhaps
structured metadata attached to each VM tag), which one do you want?". And
then have the ability for the client ot pick. But you can't really do this
sanely with just extensions, I think this is a good candidate for actually
extending the base protocol.
I have seen KVM switches that use VNC implement an extension that is
basically, SetName. They then use a client encoding to switch the KVM.
This is definitely an interesting area to pursue.
The idea is that on localhost, the server can render directly to a shared
memory segment that the client also maps. If the one is smart about how
this shared memory segment is chosen, it can be used as an XShmImage.
Yeah we did the exact same thing, with a slight twist. We get the server to
allocate the SHM segment, and the client just attaches to it. If you do it
this way, you don't need a new client->server message, the server just sends
the client the shm id if the client supports the "SharedFB" extension (that's
what we named it).
I chose to have the client allocate the memory since it knew the X
server's line size which is required to allocate an XShmImage. My first
implementation had the server allocate the buffer but required the
client to tell the server what the line size should be.
There's some complexity involved with how the server can actually figure out
whether the client is on the same machine (e.g. if the client is tunneling
over SSH, this is not easy to do). So in fact what we do is just if the
client supports SharedFBInfo and SharedFBRect, the server just sends along
the messages. The client then tries to map the segment described in
SharedFBInfo. If it fails, it just sends a new encoding set with those
encodings removed, and the server starts sending real pixel data instead.
I've thought about this problem too. My fear is that one could
accidentally map a shared memory area that's actually valid.
Yes, this means that we always try and fail sharedfb so there's a couple
extra roundtrips before remote clients actually get real pixel data. This was
deemed acceptable (by me ;-) for our use cases since most networks are fast
enough. But if this is too hackish for your taste we could bake in something
I really haven't thought of a better solution myself. That doesn't mean
one exists though.
I believe http://wiki.multimedia.cx/index.php?title=VMNC
covers some of
your extensions right? Have you gotten the appropriate encodings
reserved in the RFB spec? I don't want to support any encodings that
aren't properly registered so this would be an important first step.
So once upon a time, we tried to register encodings, but we didn't get very
far. It sounds like perhaps the process has improved since then. I will try
to get the ball rolling on this again.
I can help you with that if you want. It wasn't a painful process at
all for me.
I'm certainly happy to work toward a common set of VNC extensions to
cover virtualization. It's in everyones benefit to ensure that the
largest number of clients Just Work out of the box.
Agreed. I look forward to hearing more of your thoughts on this subject.
Same here! Thanks for the response.