Re: [Qemu-devel] [PATCH] ui/vnc: VA API based H.264 encoding for VNC framebuffer updates

[Verbeiren, David]

On 10/01/2013, 12:20 PM, Alexander Graf wrote:
> On 10.01.2013, at 12:10, Gerd Hoffmann wrote:
>> On 01/10/13 11:37, Stefano Stabellini wrote:
>>> On Thu, 10 Jan 2013, Gerd Hoffmann wrote:
>>>> On 01/09/13 23:18, Alexander Graf wrote:
>>>> Using libavcodec directly is a non-starter as distros don't ship that
>>>> due to the multimedia codec patent mess.
>>> 
>>> libavcodec is certainly available on my ubuntu and debian installs.
>> 
>> Full version with H.264 support?  Or stripped down, with patented codecs
>> removed?
>
> IIRC Ubuntu has full support. Also openSUSE has Packman where you could get
> full libavcodec builds.
>
> But overall, gstreamer is probably the better interface to look at here,
> as it actually provides a plugin architecture that people can plug their
> almost-legal codecs into.
>
> The main reason I brought this up was because I'd like to know whether
> there's a particular benefit in using vaapi directly vs using gstreamer to
> encode this :).

Thank you all for your interest in this.

First of all, this patch was meant to make it possible to experiment with
the approach without heavy lifting. As such, there are certainly other
alternatives that can be considered for the longer term.

Even though I can see big advantages in plugging into GStreamer for
streaming out the framebuffer, encoded as H.264 or other, I'm not sure
it would provide similar functionality to the experimental implementation
I provided, without significant additional efforts.

Currently, the patch implements sending the encoded data within
the "VNC channel", which means the data shows up naturally in the
VNC client.
If we instead use a separate mechanism, that GStreamer already supports,
to stream the framebuffer updates (e.g. RTP) to the client, an existing
VNC client would have to be enhanced with a full network stream input
support (+ extra bitstream parsing).
Alternatively, if we take an existing video player with network stream
support, we'd have to add to it full VNC protocol and user input support.
In both cases, this seems like a lot more effort to reach a functional
VNC client than what we've come up with.

Or maybe we could plug into GStreamer at both ends of its pipeline,
providing a GstSource at the input, feeding the pipeline with the
VM framebuffer updates, and a GstSink at the output to grab back
the encoded bitstream before sending it within the standard
"VNC channel".
We'd still have to add full bitstream parsing in the client,
but that probably makes the solution more standard anyway.

>From this point of view, interfacing to libavcodec would seem to
potentially bring some of the advantages (no need to maintain own encoder
code that would need to evolve with VAAPI) without any impact on the
VNC side.

However, I'm not sure any of those libraries currently supports
VA API encoding (only decoding, I think). I know there has been some effort
towards that but I don't think it has been finalized yet. I'll check what
the latest status is there.

Regards,
-David
Intel Corporation NV/SA
Kings Square, Veldkant 31
2550 Kontich
RPM (Bruxelles) 0415.497.718. 
Citibank, Brussels, account 570/1031255/09

This e-mail and any attachments may contain confidential material for the sole 
use of the intended recipient(s). Any review or distribution by others is 
strictly prohibited. If you are not the intended recipient, please contact the 
sender and delete all copies.