Re: [Discuss-gnuradio] [USRP-users] Gnu Radio apps freezes (locks up)

[Josh Blum]

On 11/25/2012 11:36 PM, Patrik Tast wrote:
> As reference to this issue, using F12 and *slow* dual core does not ever
> freeze even at USRP1 max sample rate. We're using *old* traditional GR
> (usrp_fft.py). Last computer + USRP1 boot was in September (boot every
> 2-3 month). We receive 24/7 2/4/8 MHz bandwidth.
> 
> Using ubuntu and udh we do see problems.
> Something *funny* has happened in graphics drivers, we suspect.
> 

The newer ubuntus enable all kinds of effects that use compositing.
After disabling every graphical "effect" on my kubuntu installation,
both unreal tournament and fft sink have much better frame rates.

-josh

> Patrik
> 
> -----Original Message-----
> From: Patrik Tast <address@hidden>
> To: address@hidden
> Cc: address@hidden
> Subject: Re: [Discuss-gnuradio] [USRP-users] Gnu Radio apps freezes
> (locks up)
> Date: Mon, 26 Nov 2012 08:42:28 +0200
> 
>> Can you confirm that its not just the gui locking up?
>> Sometimes compositioning or just funny GL setups can cause the FFT
>> plotter to have horrible rendering. Or just lower the frame rate.
> Both (GUI and rendering) lock up. I suspect the same as you "funny GL setups 
> can cause the FFT
> plotter to have horrible rendering".
> Some times the uhd_fft (using ubuntu) window changes from gray to white 
> (disabled -> enabled) every 5 sec
> 
> Patrik
> 
> -----Original Message-----
> From: Josh Blum <address@hidden>
> Reply-to: address@hidden
> To: address@hidden
> Subject: Re: [Discuss-gnuradio] [USRP-users] Gnu Radio apps freezes
> (locks up)
> Date: Sun, 25 Nov 2012 22:28:57 -0800
> 
>> However, despite this fact I cannot get rid of the "halting problem" with 
>> the 
>> UHD and the transmission freezes although I try to use the same buffer and 
>> frame 
>> length parameters as in the iperf test. See attached file. I have also tried 
>> many other combinations of the frame length and buffer size but without any 
>> luck.
>>
>> It seems like something weird is going on with UHD at high sample rates 
>> which 
>> some Ethernet cards, like in these computers, cannot handle.
>>
> 
> Can you confirm that its not just the gui locking up?
> 
> Sometimes compositioning or just funny GL setups can cause the FFT
> plotter to have horrible rendering. Or just lower the frame rate.
> 
>> Can anyone explain this inconsistent behavior or even better offer a working 
>> solution?
>> Please see the transcripts in the attached text file.
>> Also, what does "Unexpected error on recv, continuing... Error code: 1"  
>> mean or 
>> indicate?../../qr2/host/grc_examples/qr2_1ch_compass.grc
>>
> 
> Its a timeout. If something was lost due to overflow, the recv call will
> timeout trying to obtain the lost samples. Its just the way the example
> was written.
> 
> -josh
> 
>> /  Rickard
>>
>>
>> On 18 jul 2012, at 11:11, Nazmul Islam <address@hidden 
>> <mailto:address@hidden>> wrote:
>>
>>> Hello Rickard,
>>>
>>> Could you get a permanent or working solution of your problem? I think that 
>>> I 
>>> am facing a similar issue as you did.
>>>
>>> In my application, the receiver USRP senses the channel in a repeated 
>>> on-off 
>>> manner. The receiver USRP run for 200 msec, gathers the complex data in 
>>> some 
>>> files, does some processing, plots the data and waits for my input. As I 
>>> press 
>>> a button, the USRP starts again and repeats the whole process. The problem 
>>> is: 
>>> if I run the code at a very high rate (20 MS/s with 32 bit/complex sample), 
>>> sometimes the application freezes up or hangs without giving any error. 
>>> Currently, this happens within 15-20 starts. This problem occurs less 
>>> frequently if I run it at lower sampling rates (e.g. 2 MS/s).
>>>
>>> I am using USRP N210, the latest GNUradio & UHD drivers, Ubuntu 12.04. I 
>>> believe that my issues are quite similar to the ones that you faced. It 
>>> would 
>>> be great to know if you found a (somewhat) working solution.
>>>
>>> Thanks,
>>>
>>> Nazmul
>>>
>>> On Fri, Apr 13, 2012 at 10:59 AM, Rickard Radio <address@hidden 
>>> <mailto:address@hidden>> wrote:
>>>
>>>     On Apr 10, 2012, at 11:55 PM, Nick Foster wrote:
>>>
>>>>     On Tue, Apr 10, 2012 at 1:16 PM, Rickard <address@hidden
>>>>     <mailto:address@hidden>> wrote:
>>>>
>>>>
>>>>         On 31 mar 2012, at 15.43, Tom Rondeau wrote:
>>>>
>>>>         > On Thu, Mar 29, 2012 at 6:04 AM, Rickard Radio
>>>>         <address@hidden <mailto:address@hidden>> wrote:
>>>>         >>
>>>>         >> On Mar 29, 2012, at 3:26 AM, Tom Rondeau wrote:
>>>>         >>
>>>>         >>> On Wed, Mar 28, 2012 at 10:02 AM, Rickard Radio
>>>>         <address@hidden <mailto:address@hidden>> wrote:
>>>>         >>>> Hi list,
>>>>         >>>>
>>>>         >>>> After I upgraded to latest Gnu Radio 3.5.2, and latest UHD 
>>>> (and
>>>>         images), GR applications just freeze when running. No warnings, 
>>>> error
>>>>         messages or overflows etc. Just freeze.
>>>>         >>>> A simple FFT plot directly on received samples from the 
>>>> USRPN210
>>>>         just freezes after some seconds, or minutes (depending on the 
>>>> sample
>>>>         rate), although the load on the machine is not high. Need to kill
>>>>         GR-app and restart it, with the same problem occurring again.
>>>>         >>>>
>>>>         >>>> This has never been a problem with earlier versions of GR/UHD,
>>>>         from about 6 months ago.
>>>>         >>>> The freezing happens quicker with high sample rate setting but
>>>>         also with lower, eventually. No overflows happen (which was 
>>>> possible
>>>>         to get before with too high sample rates or load, etc.)
>>>>         >>>>
>>>>         >>>> The USRPN210 stops sending samples to the computer at the same
>>>>         moment as GnuRadio freezes (as observed on the system monitor).
>>>>         >>>>
>>>>         >>>> Same thing happen on two identical laptops running Ubuntu 
>>>> 10.10
>>>>         (also upgraded it from 10.04). Not sure if its a strict GnuRadio
>>>>         problem (since it worked before), UHD, or some problem with the
>>>>         Ubuntu Linux 10.10. It work(ed) flawlessly with another machine on
>>>>         OSX (before I tried to upgrade GR on it but then got stuck...) with
>>>>         identical UHD version and images.
>>>>         >>>>
>>>>         >>>> Installation of UHD+GnuRadio with the automatic linux script
>>>>         runs without any problems, as before, no errors or warnings.
>>>>         >>>>
>>>>         >>>> Any de-freezing help or clues appreciated!
>>>>         >>>>
>>>>         >>>> Rickard
>>>>         >>>
>>>>         >>> Rickard,
>>>>         >>>
>>>>         >>> Just to be clear. When you install 3.5.2 from the tarball, it
>>>>         freezes.
>>>>         >>> When you use the build-gnuradio, everything works fine?
>>>>         >>>
>>>>         >>> What's your machine?
>>>>         >>>
>>>>         >>> Tom
>>>>         >>
>>>>         >> Tom,
>>>>         >>
>>>>         >> The installation with build-gnuradio script works just fine, as
>>>>         before (no tarballs).
>>>>         >> Same result on both laptops, Acer Aspire TimelineX with i3
>>>>         processors (2.26 GHz),  running Ubuntu 10.10.
>>>>         >> I did not have this problem earlier with Ubuntu 10.04. Or on a 
>>>> Mac
>>>>         with OS X (with the source from git).  Could Ubuntu 10.10 cause the
>>>>         problem somehow?
>>>>         >>
>>>>         >> Note: Halting/freezing only happens when running an application
>>>>         (with the N210) as a receiver. (Not transmitter, see below.)
>>>>         >> The flow of receiving samples just halts after a while and the
>>>>         application freezes/halts (as a consequence).
>>>>         >> This happen sooner with high sampling rate (after a few seconds
>>>>         with 25MSPS), but eventually also with a bit lower sample rates. 
>>>> The
>>>>         CPU's are not overwhelmed (< 50%).
>>>>         >> It happens even if the UHD usrp source is connected directly to 
>>>> a
>>>>         null sink only. I do not get any overflows before the halt.
>>>>         >> In fact, I cannot even provoke a continuos stream of overflows
>>>>         since the reception just halts instead of producing overflows, 
>>>> which
>>>>         was the result earlier.
>>>>         >>  GnuRadio itself does not freeze as a whole (like the grc in the
>>>>         background), just the running application, which I then need to 
>>>> abort.
>>>>         >>
>>>>         >> Strangely enough, this "freezing/halting" does NOT happen when
>>>>         transmitting, correspondingly, even with high transmit sample rates
>>>>         such as 25 MSPS (or now possible even with 50MSPS with 8
>>>>         bit/samples!). Then it works just fine - even without underruns 
>>>> (when
>>>>         using just a file source).
>>>>         >>
>>>>         >>
>>>>         >> / Rickard
>>>>         >
>>>>         >
>>>>         > Rickard,
>>>>         >
>>>>         > Thanks for the data. Unfortunately, I have no idea what to make 
>>>> of
>>>>         > this. There isn't that much difference between the last release
>>>>         > (3.5.2.1) and what you get using the build-gnuradio script. That 
>>>> just
>>>>         > grabs the latest master version from our git repo, and we 
>>>> haven't done
>>>>         > all that much since the release. That really doesn't make a lot 
>>>> of
>>>>         > sense.
>>>>         >
>>>>         > How's your git? If you're comfortable doing so, can you check 
>>>> out the
>>>>         > v3.5.2.1 tag on git and try that one instead of the tarball 
>>>> release?
>>>>         >
>>>>         > Thanks,
>>>>         > Tom
>>>>
>>>>
>>>>
>>>>         Thanks for your info.  My current take on this interrupt issue:
>>>>
>>>>         The problem may not be gnuradio or uhd's "fault", I now suspect.
>>>>         Instead somehow the network connection and its settings might cause
>>>>         the interrupts. However, I am no linux guru so I am learning at the
>>>>         same time as I am doing.
>>>>
>>>>         First, I've updated Ubuntu, from release 10.10 to 11.04, but then
>>>>         nothing worked (just got a completely blank screen without any 
>>>> gui),
>>>>         so fast-forward upgraded to 11.10 and then both laptops came right
>>>>         back on track. I then also updated the gnuradio+uhd to latest 
>>>> (3.6.0)
>>>>         version using the excellent build-gnuradio script (as before), 
>>>> which
>>>>         itself went just fine (also as before).
>>>>
>>>>         Unfortunately, the exact same problem with interrupts (total halt) 
>>>> in
>>>>         the receiver at high sample rates persists. Note, as before, this
>>>>         happens (only?) with very high rates over the Ethernet (about 400
>>>>         mbit/s or more, the higher rate the sooner it halts, typically 
>>>> just a
>>>>         few secs) although the computer display no overruns or other 
>>>> errors.
>>>>
>>>>         Then by jacking around with the MTU setting (100,500, 1500,5000,
>>>>         etc), increasing the default too low (and initially also gnuradio
>>>>         complaining) buffer settings (net.core.rmem_max, 
>>>> net.core.wmem_max),
>>>>         disabling the Ubuntu network manager (via the menu) and removing 
>>>> the
>>>>         automatic network configuration when USRPN210 connects and instead
>>>>         setting up the network connection manually with "sudo ifconfig eth1
>>>>         192.168.10.1" ,  I *sometimes* can get the Ethernet connection 
>>>> into a
>>>>         state to work with the N210 at high sampling rates without any
>>>>         interruptions at all !
>>>>
>>>>         In that case, a beautiful continuous flow of samples to the 
>>>> crunching
>>>>         computer (like a fft-plot), at highest possible rate 50 MSPS, 8
>>>>         bit/samples over the wire. This *can* happen with a MTU above 1500
>>>>         (or more), buffers increased to recommended settings by UHD, and 
>>>> when
>>>>         this works the Ubuntu system manager tells me that some 834
>>>>         Mbit/samples flows through the Ethernet cable, and about 50% load 
>>>> on
>>>>         the CPU-cores, very nice. Then it also works for repeated runs, not
>>>>         just one "lucky" one, and for a prolonged period of time (more than
>>>>         an hour).  In the working network state I can also easily provoke
>>>>         nice expected overruns, strings of 'ooooooooooo':hs, which isn't
>>>>         possible when the Ethernet connection is in the "wrong" and
>>>>         "interrupted" state - since then it just halts/stops without 
>>>> further
>>>>         info.
>>>>
>>>>         However, finding this working network state is not just a matter of
>>>>         setting the particular network parameters as I hoped it to be...  I
>>>>         suspect some other things are happening behind the scenes, maybe 
>>>> some
>>>>         other settings etc (I do not yet have full knowledge of ethernets
>>>>         full functionality and behavior, there may be more influencing
>>>>         parameters ?) which prevent me finding the working network state 
>>>> in a
>>>>         consistent manner. Quite weird.
>>>>
>>>>         I have USRP-N210 rev2 (says sticker on the back) but now noticed 
>>>> that
>>>>         when I burned the latest fw and fpga images with the net burner 
>>>> tool
>>>>         it prints "Hardware type: n210_r3" although I selected the fpga
>>>>         version image for rev2 corresponding to my version. Could this be
>>>>         related to my issue? Haven't noticed that inconsistent message
>>>>         earlier, though.
>>>>
>>>>         If some of this rings any bells, please give me some further 
>>>> advise.
>>>>         Sorry for long post.
>>>>
>>>>         Thanks,
>>>>         Rickard
>>>>
>>>>
>>>>     My first guess is that network-manager sucks, and it's cutting your
>>>>     connection. I've seen this on Ubuntu on a semi-regular basis even on my
>>>>     own laptop, when using ifconfig to set the adapter address manually,
>>>>     sometimes network-manager decides to bring the interface down just
>>>>     because. To diagnose this, look to see the network light on the N210 go
>>>>     out after the samples stop coming. The solution is to go into the 
>>>> network
>>>>     configuration and set eth0 (or whatever network adapter it is) to use a
>>>>     static IP address instead of "auto".
>>>>
>>>>     --n
>>>
>>>
>>>     Thanks for your suggestions.
>>>
>>>     Yes, this sounds quite related to what I also experience. But in this 
>>> case
>>>     when the samples stop coming, the orange led on the N210 (upper right
>>>     side) goes from constant light up to a high frequency blinking, until I
>>>     abort the gr-script and the light goes completely out. The IP address
>>>     typically stays put, so I don't have to re-enter it with ifconfig again
>>>     after the samples stops coming. Sometimes, however, it happen what you
>>>     explain (it loose its ip address) but it seems to be uncorrelated with 
>>> the
>>>     sudden interrupts I have...
>>>
>>>     I have tried what you suggest to disable the automatic connection in the
>>>     network manager. It seems to become a bit better, the stop usually comes
>>>     later (its a random phenomenon), but eventually it stops nevertheless.
>>>
>>>     It might be the actual hardware driver for the Ethernet card which is
>>>     causing the troubles, but further diagnosis is necessary for my patient.
>>>      Found on "the internet" a thread describing quite similar symptoms as 
>>> my
>>>     patient suffer from:
>>>     http://ubuntuforums.org/showthread.php?t=1890369
>>>
>>>     The Ethernet interface is labeled "Atheros Communicaitons AR8151 v1.0
>>>     Gigabit Ethernet" but I haven't found the appropriate linux drivers and
>>>     how to install those yet.
>>>
>>>     / Rickard
>>>
>>>
>>>     _______________________________________________
>>>     Discuss-gnuradio mailing list
>>>     address@hidden <mailto:address@hidden>
>>>     https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>>
>>>
>>>
>>>
>>> -- 
>>> Muhammad Nazmul Islam
>>>
>>> Graduate Student
>>> Electrical & Computer Engineering
>>> Wireless Information & Networking Laboratory
>>> Rutgers, USA.
>>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Discuss-gnuradio mailing list
>> address@hidden
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>
> 
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
>