[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Discuss-gnuradio] inefficient large vectors

From: Marcus Müller
Subject: Re: [Discuss-gnuradio] inefficient large vectors
Date: Thu, 22 Aug 2013 21:02:28 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130623 Thunderbird/17.0.7

Hi Miklos,

with sync blocks and fixed rate decimators/interpolators, the scheduler inherently knows how many buffers to allocate etc down the signal processing line to always keep all blocks busy. With general blocks, this is not possible; calls to forecast are necessary to determine how much data needs to be supplied to keep the signal processing chain running. I'm not quite sure if there is a performance penalty for blocks that just forecast a need for as many samples as they're asked to produce or if they are scheduled identically to sync blocks; Documentation on the original GR scheduler is really really sparse and the code itself is your primary source of help... I really can't give you any hints, as I've (most of the time) tried to get along without going too deep into the GR scheduling framework (and hoping for something more readible to come along for the next major release of GR :) )

Happy Hacking

Hi Marcus,

On Thu, Aug 22, 2013 at 12:00 PM, Marcus Müller <address@hidden> wrote:
Just to add my two cents:
Depending on your actual application, your large vectors might not actually
quite fit the idea of "streams"; they might, for example, be a valid,
decoded network packet or something of the like. If they don't need
sample-synchronous handling, using messages to pass them around might work
Downside of that is of course that you can't use your favourite GR block on
messages. You break the sample synchronous architecture of a flowgraph with
multiple paths from source(s) to sink(s); and if you convert from message to
stream and back, you basically lose the vector attribute of your data (or
run into the same problems as before).
I can't really tell you much about computational performance of passing
around large messages, however.

On the other hand, you can reduce your per-block coding overhead for
Martin's suggested tag-based solution:
Write a base class that implements a input and output buffer and a minimal
state machine based on stream tag evaluation. Let your blocks inherit from
that. Always copy as many items from your general_work's input vector to
your input buffer as you can, and copy as many samples from the output
buffer to your general_works output vector as possible. Execute your
computation when your input buffer is full and your output buffer empty.
That way, you'll get a quasi-fixed relative rate, but get all the freedom
and scheduling disadvantages of a general_work block with an itemsize of
gr_complex (or whatever your data type is).

I know from experience that this might be hard to debug. However, once your
state machine is watertight, you're not very likely to run into issues
Thank you for the excellent advice. I did not thought of a generic
base class, which might help me. There is one block that produces one
vector of bytes (packet data) and one vector of ints (number of errors
corrected and uncorrected), which would be impossible to solve with a
stream since the output rate is not the same. Other than this, I think
your suggestion would work.

You say that there are scheduling disadvantages of a general_work
block. What are they? Sometimes I am running into issues with the
scheduler, but it is not clear how it really works, what should I try
to do and what should I try to avoid. Can you describe it in a few
words or give a pointer where I can read upon the technical (!!)


Happy hacking,

On 08/21/2013 07:59 PM, Miklos Maroti wrote:

I have many sync blocks that work with large fixed size vectors, e.g.
converts one vector of size 12659 to another with size 18353. I have
just multiplied the sizeof(gr_complex) with 12659 and 18353 in the
signature. However, when the flow graph is running, then I get a
warning about paging: the circular buffer implementation allocates
large buffers (e.g. 4096 many to make the paging requirement). I do
not want really large buffers. I have implemented the whole thing with
padding, but that becomes also really inefficient, since when you want
to switch between vectors and streams, then you have to jump through
extra hoops with the padding. In a previous version I had streams
everywhere, but then there is absolutely no verification whether I
messed up the sizes of my "virtual vectors".

So is there a way to work with large odd length vectors which does not
have this buffer allocation problem, and does not require padding? It
seems to me that it could be supported: regular streams but the vector
size would be verified separately at connection time and would not be
used to multiply the item size. Any advice is appreciated...


Discuss-gnuradio mailing list

Discuss-gnuradio mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]