Re: file_sink and performance monitoring

[Jeff Long]

As you noticed, the scheduler currently has some corner cases, and changes to the flowgraph can affect which blocks see backpressure or starvation. We recently noticed that interpolators can become hogs even at very low rates for this reason.

The file sink may benefit from an output multiple. The scheduler does not allow changing of output multiple in a running flowgraph, so this could cause output to be truncated to a multiple. There is no guarantee that all samples make it through a flowgraph at termination, so this may not matter. The main reason I imagine file sinks affect performance (other than just adding another block) is that they write to disk.

Yes, you may find blocks where the buffer is never full. This isn't wrong, and it tells you that that block isn't causing backpressure or seeing it from downstream blocks.

On Tue, Aug 31, 2021 at 12:09 PM Jim Melton <jim.melton@sncorp.com> wrote:

GNU Radio 3.8.2.0, CentOS 7.  NDR-358 front end.

I am working on a modification to GNSS-SDR (which is an application built upon gnuradio). It has the option to add a file_sink as a consumer of nearly every block. However, if I enable dumping in one of the upstream blocks, the performance downstream changes, as if data is being dropped so the demodulated signal becomes incoherent.

 

Conventional wisdom is that file_sink is very slow, so I can imagine that adding it to my flowgraph causes things to back up enough that we can’t keep up with the real-time input. (As an aside, file_sink could probably improve performance by setting an output multiple so it is called less frequently)

 

In trying to troubleshoot/characterize this situation, I added a monitor thread to periodically dump buffer stats for my signal source (it was originally monitoring a different block, which is why it prints both input and output buffer stats):

 

void stats_thread(gr::block* item)

{

    while (true)

    {

        std::this_thread::sleep_for(1s);

        LOG(INFO) << "source: "

                  << "\n    input  0 full: " << item->pc_input_buffers_full(0) << "%"

                  << "\n    input  0 avg:  " << item->pc_input_buffers_full_avg(0) << "%"

                  << "\n    input  0 var:  " << item->pc_input_buffers_full_var(0) << "%"

                  << "\n    output 0 full: " << item->pc_output_buffers_full(0) << "%"

                  << "\n    output 0 avg:  " << item->pc_output_buffers_full_avg(0) << "%"

                  << "\n    output 0 var:  " << item->pc_output_buffers_full_var(0) << "%"

                  ;

    }

}

 

My output always prints 0 for each of these numbers.

 

Do these methods work? Is there some other direction I should take to identify the problem? I’m willing to write a non-blocking archiving module, but if I can’t prove I’m overflowing the buffer, I need to re-think my strategy.

---

Jim Melton

Principal Software Engineer

Sierra Nevada Corporation

 

CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are confidential, may contain proprietary, protected, or export controlled information, and are intended for the use of the intended recipients only. Any review, reliance, distribution, disclosure, or forwarding of this email and/or attachments outside of Sierra Nevada Corporation (SNC) without express written approval of the sender, except to the extent required to further properly approved SNC business purposes, is strictly prohibited. If you are not the intended recipient of this email, please notify the sender immediately, and delete all copies without reading, printing, or saving in any manner. --- Thank You.