[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Discuss-gnuradio] make test failure at qa_constellation_receiver

From: Tom Rondeau
Subject: Re: [Discuss-gnuradio] make test failure at qa_constellation_receiver
Date: Sun, 15 Apr 2012 17:46:27 -0400

On Sun, Apr 15, 2012 at 1:13 AM, Ben Reynwar <address@hidden> wrote:
> It appears that each test case is being run twice.  I think once to
> generate the xml output, and once to generate output for stdout and
> tell you whether it failed.  The random number generator isn't being
> reseeded at the start of each test so they can produce different
> results, so you can sometimes see debug statements that indicate it
> should fail, but the output to stdout claims that is passes.  I've
> attached a patch that make each test use it's own random number
> generating object which seems a tidier way to do this, and makes the
> tests repeatable.
> Since there seem to be a fair number of choices of seed that produce
> an error fraction of over 0.2, I've also increased the acceptable
> error rate in the test to 0.3, as Alick initially suggested.

Hey Ben and Martin,

Both patches have been applied. It fixed the issues on my 32-bit VM,
too, so it looks like we got it.

Thanks for help!


> On Sat, Apr 14, 2012 at 9:39 AM, Alick Zhao <address@hidden> wrote:
>> On Sat, 14 Apr 2012 13:40:35 +0200, Martin Braun wrote:
>>> On Sat, Apr 14, 2012 at 05:20:13PM +0800, Alick Zhao wrote:
>>>> Yes the bug still occurs.
>>>> I just wrote a simple script to ran the test for 50 times on T60 with
>>>> Ubuntu, GNU Radio 3.5.3 equipped, before and after FREQUENCY_OFFSET set
>>>> to 0. All failed the test. (n_pass is 0/50 in both sets) Every test's
>>>> output is almost identical in each set except particular test time
>>>> length. The ones before FREQUENCY_OFFSET change is basically the same as
>>>> test.t60.log already attached. One of the ones with FREQUENCY_OFFSET set
>>>> to 0 is attached below. I guess the almost same contents of output is
>>>> due to fixed random seed.
>>>> I also ran the script on a Dell desktop with Fedora, and the result is
>>>> 50/50 pass the test. Then I notice one line in the qa python file says
>>>> that seed 1234 fails. However, 50/50 are OK with seed 1234 on the Dell
>>>> desktop.
>>> So,
>>> I tried this on a native 32-Bit Ubuntu 11.10, and it fails (tells me
>>> it's using "Volk machine: sse3_32".
>>> I also noticed the line that says seed=1234 fails. Also, the seed is set
>>> multiple times in the script. If this is the source of the bug or not,
>>> it should be fixed, because the seed is reset somewhere in the guts of
>>> the test. I'll post a patch on patch-gnuradio--this eliminates the
>>> problem on my machine, for some reason. If it does the same for you,
>>> this might actually be the solution.
>>> MB
>> I applied the patch and tested it. No failure, yeah!
>> However, with my debugging line, I can see this:
>> [...]
>> constellation: <constellation psk (m=32)>  differential: True  correct:
>> 0.942307692308
>> constellation: <constellation psk (m=64)>  differential: True  correct:
>> 0.772435897436
>> constellation: <constellation psk (m=2)>  differential: True  correct: 1.0
>> [...]
>> So why 0.77XXX on the second line does not cause the assert to fail?
>> alick
>> _______________________________________________
>> Discuss-gnuradio mailing list
>> address@hidden
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

reply via email to

[Prev in Thread] Current Thread [Next in Thread]