[Gnash-dev] Summer of Code Status

[Jonas 'Sortie' Termansen]

Hi,

I've been working on Gnash as my Google Summer of Code project.

My task was originally optimizing Gnash's libmedia, which contained 
needless memcpy's and other needless expensive operations. I also took 
on the task of changing the AudioDecoder to a push/pop/pull interface 
(like VideoDecoder).

To get going, I started by fixing some build errors I had experienced. I 
then spent a while improving configure, such that it tells you what 
development packages you need to apt-get, and it offers to do it for 
you. I also fixed quite a few bugs in configure (requiring that 
development headers are installed to link, previously it just looked for 
the shared library, and more things like this).

Since then, I've worked on libmedia. I've already pushed a few 
optimizations to git master, which both improved and fixed 
AudioDecoderSimple. More importantly, I set out to create testcases for 
libmedia, such that we can validate that it works as intended. Since my 
main task was working on the AudioDecoder interface, I decided that my 
testcases should initially just test audio.

My testcases uncovered a few critical design problems in libmedia:

The AudioDecoder interface is not designed such that the input may be 
reused later. This is critical for the libavcodec backend, which expects 
us to re-provide the same data packet, if the data packet contained 
multiple audio frames. To solve this, calling the decode() function 
transfers ownership of the data packet to the AudioDecoder, which may 
then store the data in a queue until it has been used. In effect, this 
implements the push/pop mechanism we intended AudioDecoder to have.

Another critical design flaw is that MediaParser has no way to signal 
whether a file is not supported because 1) the codec is missing 2) the 
file is broken 3) our code is broken. Because of this, my testcases 
assume the worst and declare the file FAILed. This has caused me 
problems on our buildbots, where the needed codecs weren't installed. 
gg0 addressed this by installing most of the codecs, which means most of 
the testcases now succeed. However, the testcases still vary greatly 
depending on the system configuration and library versions, instead of 
varying depending on whether our code is actually broken.

MediaParser also has a lot of issues, such as if something went wrong 
while parsing the file, it cannot signal that something went wrong. 
Rather, it does nothing and my testcase thread must timeout after 2 
seconds and warn that something _might_ have gone wrong.

I've been hesitant to push my testcases because of these issues. While 
useful, my testcases would do nothing but spam spurious FAILes or 
XPASSes when running on the buildbots. I would like to address the 
MediaParser issues before pushing my testcases to git master. (If anyone 
wants to review my testcase code and other changes, tell me and I'll 
give it to you.)

My testcases needs a lot of fixes in libmedia to work, and I have 
already addressed quite a few of them, most notably multi-packet 
decoding, proper codec detection in AudioDecoderFfmpeg, and more. I have 
already pushed some of these fixes, but the multipacket support is not 
pushed yet. This is simply because my testbuild has been overly crashy 
when running Youtube - I want to be sure I don't cause random crashes. :)

So what now?

I expect to keep working on Gnash. First, I'll bugcheck my multiframe 
code and then push it to git master. Then I'll take on the task of 
reworking MediaParser, such that it can communicate failures better. 
Hopefully, this should make fix the last few failing testcases so I can 
push it.

Then I want to move media decoding to the media parsing thread. This 
offers performance benefits for libavcodec, which prefers if we decode a 
packet before we parse the next (and the previous packet is garbage 
collected). In addition, it breaks the 'one-packet one-frame' paradigm 
which the current AudioDecoder interface suffers from. Lastly, 
MediaParser will work as intended: you seek to a position in the file, 
and it returns a decoded audio/video frames with associated 
metainformation (timestamp, etc.).


So overall, most of my code has been pushed to git master, some of it 
isn't because it requires fixes in libmedia. I intend to perform these 
fixes over the coming weeks, such that my testcases can be pushed soon. 
Then I'll continue to make improvements to libmedia.

Thanks for a great summer!

Jonas 'Sortie' Termansen