Re: [Discuss-gnuradio] make test failure at qa_constellation_receiver

[Tom Rondeau]

On Sun, Apr 15, 2012 at 1:13 AM, Ben Reynwar <address@hidden> wrote:
> It appears that each test case is being run twice.  I think once to
> generate the xml output, and once to generate output for stdout and
> tell you whether it failed.  The random number generator isn't being
> reseeded at the start of each test so they can produce different
> results, so you can sometimes see debug statements that indicate it
> should fail, but the output to stdout claims that is passes.  I've
> attached a patch that make each test use it's own random number
> generating object which seems a tidier way to do this, and makes the
> tests repeatable.
>
> Since there seem to be a fair number of choices of seed that produce
> an error fraction of over 0.2, I've also increased the acceptable
> error rate in the test to 0.3, as Alick initially suggested.

Hey Ben and Martin,

Both patches have been applied. It fixed the issues on my 32-bit VM,
too, so it looks like we got it.

Thanks for help!

Tom


> On Sat, Apr 14, 2012 at 9:39 AM, Alick Zhao <address@hidden> wrote:
>> On Sat, 14 Apr 2012 13:40:35 +0200, Martin Braun wrote:
>>> On Sat, Apr 14, 2012 at 05:20:13PM +0800, Alick Zhao wrote:
>>>> Yes the bug still occurs.
>>>>
>>>> I just wrote a simple script to ran the test for 50 times on T60 with
>>>> Ubuntu, GNU Radio 3.5.3 equipped, before and after FREQUENCY_OFFSET set
>>>> to 0. All failed the test. (n_pass is 0/50 in both sets) Every test's
>>>> output is almost identical in each set except particular test time
>>>> length. The ones before FREQUENCY_OFFSET change is basically the same as
>>>> test.t60.log already attached. One of the ones with FREQUENCY_OFFSET set
>>>> to 0 is attached below. I guess the almost same contents of output is
>>>> due to fixed random seed.
>>>>
>>>> I also ran the script on a Dell desktop with Fedora, and the result is
>>>> 50/50 pass the test. Then I notice one line in the qa python file says
>>>> that seed 1234 fails. However, 50/50 are OK with seed 1234 on the Dell
>>>> desktop.
>>>
>>> So,
>>>
>>> I tried this on a native 32-Bit Ubuntu 11.10, and it fails (tells me
>>> it's using "Volk machine: sse3_32".
>>>
>>> I also noticed the line that says seed=1234 fails. Also, the seed is set
>>> multiple times in the script. If this is the source of the bug or not,
>>> it should be fixed, because the seed is reset somewhere in the guts of
>>> the test. I'll post a patch on patch-gnuradio--this eliminates the
>>> problem on my machine, for some reason. If it does the same for you,
>>> this might actually be the solution.
>>>
>>> MB
>>>
>>>
>>
>> I applied the patch and tested it. No failure, yeah!
>>
>> However, with my debugging line, I can see this:
>>
>> [...]
>> constellation: <constellation psk (m=32)>  differential: True  correct:
>> 0.942307692308
>> constellation: <constellation psk (m=64)>  differential: True  correct:
>> 0.772435897436
>> constellation: <constellation psk (m=2)>  differential: True  correct: 1.0
>> [...]
>>
>> So why 0.77XXX on the second line does not cause the assert to fail?
>>
>> alick
>>
>> _______________________________________________
>> Discuss-gnuradio mailing list
>> address@hidden
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>