|
From: | Jim Melton |
Subject: | RE: Re: Reading tags from ZMQ in Python |
Date: | Tue, 2 Nov 2021 01:14:44 +0000 |
I’m sorry, you didn’t specify that you were using the ZMQ SUB source; my answer was for a more general ZMQ interface. The behavior you are seeing is “normal” GNU Radio behavior. A block is called whenever the scheduler decides that it can produce output. The number of items it
is allowed to produce is constrained by available space in the output buffer (which is controlled by how fast downstream blocks consume). So 20 to 2000 samples at a time is expected and normal behavior. Tags are … interesting. The tags in range of the samples send are serialized into a string that is pre-pended to the ZMQ message. Personally, I find this decision unfortunate. I probably
would have sent a multi-part message with the IQ first and each tag as a separate part, PMT-serialized. As it is, you can look in tag_headers.cc for the
gen_tag_header function. The format of this header appears to be: ·
A 16-bit unsigned integer magic number (sending host byte order) ·
An 8-bit unsigned integer version number ·
A 64-bit unsigned integer stream offset (sending host byte order) ·
A 64-bit unsigned integer number of tags (sending host byte order) ·
Each tag, as a 3-tuple of PMT serialized values (key, value, srcid) Now, I don’t know if it is exposed to Python or not, but there is a corresponding
parse_tag_header that is designed to pull this thing apart and return how many bytes it consumed. That’s your generalized solution. --- Jim Melton From: Temir Karakurum <temirkarakurum@gmail.com>
Hi Jim, Thanks for your prompt reply, I really appreciate it. I understand that ZMQ in a way defines a protocol and it is on me to decode it by providing the proper interfaces. However, Gnuradio ZMQ SUB Source for instance
can read whatever arbitrary tags I throw at it if I simply turn on the "Pass Tags" flag to yes. I tried to understand how it is done by investigating the source code but I don't really know c++ or zmq that well so I was a little lost. I don't know whether
it would be possible to reproduce such behavior in pure Python, but if you have any suggestions or pointers there I would love to hear them. I also realized that when I am receiving say 10000 sample long IQ vector from ZMQ, sometimes the messages arrive in 2 parts (say 8000, 2000) and sometimes in many parts where some parts can be as small as 20 long.
Is this behavior controlled by the OS kernel or is this something that can also be controlled by me. Also when cutting the incoming messages manually I usually chop the pieces guided by intuition and if things don't align, I just re-cut the piece to the right or to the left until it decodes properly. Is there any
rule of thumb regarding how these messages are constructed? For instance, how long is the header for both the first piece and the following pieces. If I have a gr.tag_t() where the tag value is defined through pmt.from_double, pmt.from_long or pmt.to_pmt
is there a simple rule to determine how long the corresponding bytes should be to properly decode via pmt.deserialize_str? Best, Temir On Fri, Oct 29, 2021 at 12:31 AM Jim Melton <jim.melton@sncorp.com> wrote:
CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are confidential, may contain proprietary, protected, or export controlled information, and are intended for the use of the intended recipients only. Any review, reliance, distribution, disclosure, or forwarding of this email and/or attachments outside of Sierra Nevada Corporation (SNC) without express written approval of the sender, except to the extent required to further properly approved SNC business purposes, is strictly prohibited. If you are not the intended recipient of this email, please notify the sender immediately, and delete all copies without reading, printing, or saving in any manner. --- Thank You.
|
[Prev in Thread] | Current Thread | [Next in Thread] |