Re: [Discuss-gnuradio] Streaming IQ File Compression

[Marcus Müller]

Well, having wasted a couple of hours of sleep time on this, maybe you shouldn't do the logarithmic storage... at its core its the same idea as storing floating point, but with a fixed mantissa.

The mathematical effects of the "rounding to the nearest exponential step" on superimposed sinusoids of different amplitude are … funky.

Best regards,

Marcus

On 16.07.2016 21:57, Juha Vierinen wrote:

Can you reduce the number of bits that you are using?

With radar signals, the receiver noise most of the time excites only about 8 bits out of 16. Ground clutter or meteor echoes excite nearly all of the bits occasionally, so I can't just truncate to 8 bits. In this case, bzip2 actually does a pretty good job of getting rid of the 8/16 most significant bits that are zero most of the time. Thus, I get a compression ratio close to 50% when using sc16. pbzip2 is a good tool for doing parallel compression on files. 

juha

On Sat, Jul 16, 2016 at 3:59 AM, Dave NotTelling <address@hidden> wrote:

Anyone have experience with streaming 100+ MSPS of 16 bit IQ data through a compression tool and then to disk?  I'd like to be able to take really wide band snaps for several minutes.  Currently that would take up 16 * 2 * SAMP_RATE bits per second.  So, for 200 MSPS that would end up being 800 MBytes/s.  That rate eats up a hard drive pretty quickly.  Running it through gzip by way of pigz was my first thought, but even on an 8 core Intel machine pigz just can't keep up.  I can sustain maybe 300 MBytes/s but that's it.  And I know it's not a hard disk limitation as the M.2 drive I am using can easily sustain 800 MBytes/s of uncompressed data.

Thoughts?

Thank you!

-Dave

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio