[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Having trouble with C++ OOT block in restricting output to those inp

From: Martin Luelf
Subject: Re: Having trouble with C++ OOT block in restricting output to those input values I wish to pass
Date: Sun, 13 Sep 2020 10:51:51 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0

Hi George,

noutput_items is a number given to you by GNURadio. I would not overwrite this variable, otherwise you no longer know how many items you can write into the output buffer. Create a new variable nitems_written or similar to track how many items you have written.

Also have a look at what each variable is tracking exactly. cnt seems to be the number of input bytes you have checked for the preamble. But you also use it to compute the position in the out array, which does not make sense to me. cnt would increase with more spurious bytes in front of the preamble, but the output position should not depend on the number of spurious bytes. From the code you showed me cnt also seems to persist between multiple calls to general_work, but the out array is always clean when general_work is called again (meaning that on every general_work call you should always start writing to element 0).

You loop over your entire input array and if the preamble was found you copy the entire preamble and then you copy every following byte one by one. That means you copy the entire preamble again (without the first byte) and if your input buffer is longer than your message size you also copy more bytes to the output than your message is long which could result in missing the next preamble. Also you are potentially writing more items into the out array than GNURadio has space for, which is why I recommend you not to overwrite noutput_items and while you are still learning check that your computed indices for out and in are within the allowed bounds before every single write and read (i.e. by using assertions).

What messages are you using in your unittest and what kind of spurious bytes? Is is all zeros, or all ones? I would recommend you use different bytes, e.g. an increasing counter for your message and a decreasing counter starting from 255 for your spurious bytes so you can quickly spot if your messages are cropped, or off in some other way. Because the way you describe your results it seems like one message is copied twice. I could not find any obvious bug in your code that would output a message exactly twice, so I suppose it copies some data that look like a message on the first glance.


On 11.09.20 03:06, George Edwards wrote:
Hi Martin,

Thanks for your detailed answer. I really appreciate the great effort you put into explaining how things work. I am still on the learning curve.

I used your suggestions to the best of my understanding and it worked beautifully for the one sync pattern test vector in the QA test.

Then, I took your suggestion for repeated sync patterns using an init flag which I reset to 0 to restart the process. For the QA test, I repeated the original input data twice (now I have 2 sync patterns), so the expected QA output should be the original output repeated twice. I modified the C code by adding an if statement at the end to check if noutput_items == Buf_size+message_size (buf_size is the length of the pattern, which I call preamble) and if it is, I reset the init flag to zero as well as other params used in the initialization section of the code. The QA test failed by producing an output with 3 repeated copies of the original output rather than the expected 2 copies. I do not expect you to send too much time looking at my code below, however, I would appreciate it very much, if you would glance at it to see if you can spot what I am doing wrong. The test to un-initialize (setting init to 0) was done towards the end of the code block after the consume method.

       int kk = 0;____

       for (int i = 0; i < ninput_items[0]; i++)____



             cnt += 1;                   // cnt number of passing bytes____

             kk = initialize(in[i]);____

             if (kk == 0){____

                 noutput_items =  cnt;____


                 memcpy((void*)out, (const void*)preamble, buf_size);____

                 noutput_items = cnt;____


          } else {

            out[i-cnt+buf_size] = in[i];____

            noutput_items = buf_size + message_size;____


       } ____

       consume_each (noutput_items);____

       if (noutput_items == buf_size + message_size){____

          init = 0;      // re-initialize all____

          cnt = 0;____

          kk = 0;____


       return noutput_items;____


Thanks very much for the help.


On Thu, Sep 10, 2020 at 12:06 AM Martin Luelf <mail@mluelf.de <mailto:mail@mluelf.de>> wrote:

    Dear George,

    this also caused me a lot of headache when I started with GNURadio, so
    here is what I learned about it.

    Let's start with the forecast method. This tells GNURadio how many
    samples you need on each input stream to generate the requested number
    of output items. Usually GNURadio will run your forecast function a
    couple of times with different output numbers to find a good data chunk
    size to give to the general work function. Keep in mind that the number
    of required input items you give here is a minimum number and GNURadio
    might decide to give you more input data than you requested. It is also
    important to know that the number of samples you request here is
    just an
    estimate. You are not forced to actually process that much data.

    Now to the general_work function. noutput_items tells you how many
    samples GNURadio would like you to produce (this is a maximum number
    may produce less). It also tells you how much memory is allocated in
    every array in the output_items vector. If you have only one output and
    you used the default <out type> *out = (<out type> *) output_items[0];
    definition this tells you how many items you can place (at most) into
    the out array.

    The ninput_items[] array tells you how many input items are
    available in
    the input arrays. Again if you just have one input and you use const
    type> *in = (const <in type> *) input_items[0]; ninput_items[0] is the
    number of inputs available in the in array. You may not read more items
    than that from the array.

    Within the given input symbols you can start looking for your sync
    pattern. If you generate output you have to write (in your case
    copy) it
    to the out array. At the end of general_work you call consume(0, K)
    the number of input items K that you have consumed on input 0. That is
    how many of the input items you have used and do not need to see again.
    If you consume 0 symbols the next call to general_work will show you
    exact same input samples again. If you consume ninput_items[0] you will
    see completely new input samples on the first input the next time
    general_work is called. And then you return the number of samples you
    generated (i.e. how much samples you put into the out array). This
    number must be smaller or equal noutput_items, because otherwise you
    would have written out of the allocated memory which might result in a
    segfault/core dump. Note that you don't have to call consume at the
    end of work and there is also another way of telling GNURadio how many
    samples you have produced, but let's leave that for another day.

    So a very easy (but not the most efficient) setup for your problem
    could be:
    Add a boolean flag to your _impl class that both forecast and
    general_work can read/write to. This flag will indicate whether or not
    you have found the sync pattern or not. You initialize this flag
    with false.
    Assume you have a sync pattern of length L and a message with M data
    symbols afterwards.

    In forecast if the flag is set (meaning you have found the sync
    you need L+M symbols of input. If the flag is not set you need L input
    samples, regardless of how many output samples GNURadio wants you to

    In general work if the flag is false you search the input for the sync
    pattern. If you found it at position i (counting from 0) you set the
    flag to true, consume i samples (i.e. everything before the sync
    marker). If the sync marker is not found you keep the flag to false and
    consume the inputs that you have searched so far. In both cases you
    return 0 since you have not generated any output yet.

    If the flag is true you copy the first L+M samples from the input to
    output, you set the flag to false (because after the data you have to
    start searching for the sync marker again) you consume L+M samples and
    return L+M samples.

    Note: This is a very easy to understand scheme, but unfortunately not
    very efficient. You only process a single block of either unwanted
    spurious symbols, or one sync marker and data at a time. So once you
    have a good understanding of how this works you should tweak that block
    to be able to process multiple blocks of spurious symbols and sync
    patterns/data within once call to general_work. It uses the same
    kind of
    logic, but requires more housekeeping of counters and indices.

    If your input is symbols rather than bits/bytes you should also look at
    the paper from J. Massey "Optimum Frame Synchronization" from 1972 on
    how to perform optimum sync marker detection, which performs better
    the intuitive correlation search.

    Hope that gets you started.


    On 10.09.20 04:34, George Edwards wrote:
     > Hello,
     > I am writing an OOT block in C++ that receives a sequence of
    numbers and
     > searches through for a sync pattern and passes to its output the
     > pattern and the  bytes of data which follows it. The QA test
    shows the
     > block recognizes the pattern and will pass the pattern along with
     > data that follow it, but there is a problem. The block does not
    know a
     > priori the number of spurious bytes preceding the sync pattern of
     > so I cannot set up the true relationship between the ninput_items
     > noutput_items. However, the block can count the number of bytes that
     > came in before the pattern. This is my problem:
     > 1) In the OOT general_work method: If I set noutput_items =
     > ninput_items[0], then in addition to passing the correct data to the
     > output, it passes trailing zeros equal to the number of spurious
     > that entered before the pattern.
     > 2) If I set the return noutput_items = ninput_items - cnt (where
    cnt is
     > the number of spurious bytes before the pattern) depending on
    where I
     > put noutput_items in the code, it either throws a core dump or
    cuts off
     > the number of true data.
     > Also, within the forecast method, I simply use a _for_ loop and set
     > ninput_items_required[i]=noutput_items;
     > I will appreciate any help to point me in the right direction in
     > with this non-regular output/ inputs relationship.
     > Thank you!
     > George

reply via email to

[Prev in Thread] Current Thread [Next in Thread]