[Gnash-dev] VA API support

[Gwenole Beauchesne]

Hi,

Since support for VA API is not to be integrated at this time, in the 
current state, I have placed the patches and instructions there: 
<http://www.splitted-desktop.com/~gbeauchesne/gnash-vaapi/>

The patch applies against bzr rev 11404.

Some background. VA API is an open spec and there is only one Open Source 
implementation at this time. Another one, with more codecs, should emerge 
by Q4 2009.

VA API, as other video decoding acceleration API on Linux (and on other 
proprietary OS), is actually a decode+display API. That is, decoded frames 
reside in GPU memory and are to be displayed from there. There are 
interfaces to get the pixels back (vaGetImage()) but it's suboptimal.

To make things worse, Flash requires RGB24 format so an extra color space 
conversion is required. Shouldn't this conversion be necessary, the 
vaGetImage() approach can be bearable, as VLC devs shown. A 1080p H.264 
video can be decoded+vaGetImage()'ed with 30% CPU load, whereas a direct 
display with vaPutSurface() would make it less than 1%. Pure SW decoding, 
with Xv output, leads to 80% CPU load. The CPU was a dual core Opteron.

The current approach, HW decoding + reading pixels back + converting to 
RGB24, is obviously too slow to be profitable.

A quick test with this video (720x576)
<http://samples.mplayerhq.hu/V-codecs/h264/NeroAVC.mp4>

Gnash was built with Gtk GUI and AGG renderer.

Average CPU load 35%. Of which, top players were:
9107     20.5645  (no location information)   no-vmlinux               
no-vmlinux               (no symbols)
8281     18.6993  (no location information)   [vdso] (tgid:6492 
range:0xffffe000-0xfffff000) gtk-gnash                (no symbols)
6712     15.1564  agg_span_image_filter_rgb.h:122 gtk-gnash                gtk-gnash 
agg::span_image_filter_rgb_bilinear<agg::image_accessor_clone<agg::pixfmt_alpha_blend_rgb<agg::blender_rgb_pre<agg::rgba8, agg::order_rgb>, agg::row_accessor<unsigned char> > >, agg::span_interpolator_linear<agg::trans_affine, 8u> >::generate(agg::rgba8*, int, int, unsigned int)
5618     12.6860  (no location information)   libswscale.so.0.7.1      
gtk-gnash                (no symbols)
1424      3.2155  agg_renderer_scanline.h:163 gtk-gnash                gtk-gnash                void agg::render_scanline_aa<agg::scanline_u8_am<agg::alpha_mask_u8<1u, 0u, agg::one_component_mask_u8> >, 
agg::renderer_base<agg::pixfmt_alpha_blend_rgba<agg::blender_rgba_pre<agg::rgba8, agg::order_bgra>, agg::row_accessor<unsigned char>, unsigned int> >, agg::span_allocator<agg::rgba8>, 
agg::span_image_filter_rgb_bilinear<agg::image_accessor_clone<agg::pixfmt_alpha_blend_rgb<agg::blender_rgb_pre<agg::rgba8, agg::order_rgb>, agg::row_accessor<unsigned char> > >, agg::span_interpolator_linear<agg::trans_affine, 
8u> > >(agg::scanline_u8_am<agg::alpha_mask_u8<1u, 0u, agg::one_component_mask_u8> > const&, agg::renderer_base<agg::pixfmt_alpha_blend_rgba<agg::blender_rgba_pre<agg::rgba8, agg::order_bgra>, 
agg::row_accessor<unsigned char>, unsigned int> >&, agg::span_allocator<agg::rgba8>&, agg::span_image_filter_rgb_bilinear<agg::image_accessor_clone<agg::pixfmt_alpha_blend_rgb<agg::blender_rgb_pre<agg::rgba8, 
agg::order_rgb>, agg::row_accessor<unsigned char> > >, agg::span_interpolator_linear<agg::trans_affine, 8u> >&)

The first two are likely sleep functions.

IMHO, the most efficient way to use HW video decode acceleration through 
an OpenGL renderer. It's possible to instruct the video decoder to render 
to an OpenGL texture, either indirectly through the texture-from-pixmap 
extension or directly. However, this would require more changes to Gnash. 
e.g. manipulating video objects with a GL texture id instead of 
transporting the decoded frames bytes (GnashImage).

Regards,
Gwenole.