|
| From: | Gonzalo Arcos |
| Subject: | Re: [Discuss-gnuradio] Discuss-gnuradio] How to specify maximum size of input buffers on blocks |
| Date: | Fri, 26 Feb 2016 14:48:34 -0300 |
Hi Gonzalo,
However i noticed that after the optimization, the sum of all blocks percentage of processing time is not 100%, this is when i started evaluating the possibility that gnu radio scheduler or gnuradio framework or the relationship between blocks is what is preventing me to achieved my desired rate, and it is not each block processing time anymore.Well, in GNU Radio, every block runs in its own thread, so you can basically can get up to the number of CPU cores * 100% as total consumption.
The performance monitor should be able to tell you how full your buffers are on average; watch out for "full" buffers: they're upstream from your bottlenecks.However, i do not have an easy way to determine the "amount of time a block has been blocked because another block in the downstream has not read the input buffer fast enough for the first block to be able to write on the buffer".
Also, look for the average rate (as reported e.g. by the probe_rate), and compare that value with the (average work items/average work call time). That kind of gives you a "block duty cycle".
Best regards,
Marcus
On 02/26/2016 01:51 PM, Gonzalo Arcos wrote:
Hi Marcus,
Thanks a lot for your answer! So now having confirmed this, comes the real question :)
I am trying to improve the performance of a flowgraph. My performance "metric" is the rate at which a file in my filestystem grows, since my flowgraph starts with a file source and ends with a file sink. This means that after doing some processing, the flowgraph outputs the result to a file, so if the file grows faster, then the flowgraph is executing faster and i achieve a faster transfer rate, even if i then change the file sink for another sink (i.e. usrp).The transfer rate is very important to me, because im evaluating the performance of the flowgraph compared to the implementation that already exists in hardware, which i know exactly the capable transfer rate.
So after using the performance monitor and detecting some blocks that do a lot of processing, ive optimized those blocks.
However i noticed that after the optimization, the sum of all blocks percentage of processing time is not 100%, this is when i started evaluating the possibility that gnu radio scheduler or gnuradio framework or the relationship between blocks is what is preventing me to achieved my desired rate, and it is not each block processing time anymore.
However, i do not have an easy way to determine the "amount of time a block has been blocked because another block in the downstream has not read the input buffer fast enough for the first block to be able to write on the buffer".
It is also not as simple as connecting a file sink in an intermediate point in the flowgraph delete the rest of the blocks, and measure the transfer rate against this new file sink. This is because in my flowgraph interpolation and decimation occurs at different blocks, resulting in making the file size an unrealiable metric in these cases.
In the only case this is valid, is at the very end of the flowgraph, when the data has been restored to its original content, and therefore, any output by from the final block corresponds to an input byte.
I hope i expressed myself correctly. Do you have any tip on how to figure out why my flowgraph at a determine rate? Should the time a flowgraph executes be always the same to the sum of the time all blocks consume?
---------- Forwarded message ----------
From: Marcus Müller <address@hidden>
Date: 2016-02-26 6:52 GMT-03:00
Subject: Re: [Discuss-gnuradio] How to specify maximum size of input buffers on blocks
To: address@hidden
Hi Gonzalo,
these are the mails I like most :)
On 25.02.2016 01:11, Gonzalo Arcos wrote:
Suppose i have a file source, connected to 2 blocks, one, lets call it Block A, does always return instantly after general work is called, in other words, it will never consume an input item. The other one, say its block B, is the identity block, it consumes and output exactly what it receives as input.
So:
The output of the first block is a null sink (since i know it will not produce any output), and the output of the second block is a file sink.
/->A->Null Sink File Source -| \->B->File SinkExactly!
What i am experiencing at running this flowgraph, is that block B will work for the very first seconds (or centiseconds), and will eventually block. This is because A has never consumed an input item, so i guess A's input buffer is full. Because of this, the file sink cannot push any more items further into the flowgraph, resulting in B not having any new input items to process.
So the mechanism below is: the output buffer of File Source is the input buffer of A and the input buffer of B. No memory duplication here.
File Source has a buffer writer with a write pointer, and A and B have their own read pointers pointing into that buffer.
When File Source produces N items, the write pointer advances by N. Similarly, when A consumes M items, A's read pointer advances by M.
When calling (general_)work, the input_items buffer(s) is (are) really just a pointer (start_of_buffer + read pointer). Equivalently, the output_items buffer(s) is (are) really just pointing to the write pointer.
File Source is only allowed to produce so many items that the write pointer doesn't advance beyond the minimum read pointer, because in that case, it would overwrite samples that a downstream block hasn't consumed.
Depends. Typically (64bit Linux), 8192 complex items get allocated, but that really depends on various factors.So, my question is, how big is the input buffer of A?
Yes! For example, as you noticed, in the GRC, open the "advanced" tab in a block property; there's a "Min Output Buffer" and a "Max Output buffer field". There's corresponding methods to set these sizes in the gr::block class[1].Is that customizable?
Yes; but that might, depending on the rate in which the file source produces samples, be *a whole lot* of memory!Can i increase the size, so the flowgraph wont block at the first seconds, but at a minute, for example?
I recommend not doing that, but instead:
- Make A consume the items it has read (because, as far as I can tell, you don't actually need them, afterwards, do you?)
- implement your "end the running of this flowgraph after X samples" by adding a "head" block, which has the functionality to simply consume all N input samples, copy them from its input to its output buffer, and then consume(N). If sum(N) == X, it says it's done and thus leads to the flowgraph be shut down
You could just write a block that does that for you.Can i specify a policy for a block that if it reaches X amount of samples in its input buffer to drop some of the input?
Because there's no such thing as a dedicated input buffer :) because every input buffer is in fact the output buffer of the upstream block.
Ive seen that each block has minimum output buffer and maximum output buffer in grc. However, i do not see any option regarding its INPUT buffer.
No, 0 just instructs GRC to not write a line containing a "set_min_output_buffer" call[2], so GNU Radio uses the default.
Another thing i thought is that A's input buffer is the file source output buffer, therefore by adjusting the output buffer of the file source, i am adjusting A's (and B's) input buffer. However, the output buffer of the file source is 0, so i guess its "infinite".
Best regards,
Marcus
[1] https://gnuradio.org/doc/doxygen/classgr_1_1block.html#a9d10e3f6747f91b215abe81b60a003d5
[2] https://github.com/gnuradio/gnuradio/blob/master/grc/python/flow_graph.tmpl#L230
_______________________________________________ Discuss-gnuradio mailing list address@hidden https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
| [Prev in Thread] | Current Thread | [Next in Thread] |