Re: [Discuss-gnuradio] Packet Transfer?

[Eric Blossom]

On Thu, Feb 09, 2006 at 08:46:53AM -0500, Michael Dickens wrote:
> After reading through the GMSK2 code, I wonder why packets are not  
> transferred in the "usual" way: via the graph "connect()" method?   
> They are instead transferred via "callforward" ["send_pkt()"] and  
> "callback" ["rx_callback()"] methods, which requires extra threads to  
> "automate" any data transfer.

Because with "the usual way" you can't see the (variable length)
packet boundaries. If you're worried about the threads, you've got
your eye on the wrong problem.  (Premature optimization...)
Specifically wrt send_pkt, you've got to solve the problem of getting
the data from python to C++.  The message/msg_queue solves it nicely.
In the past we've also used pipes and/or sockets.  FWIW the
message/msg_queue is also used to transfer data from C++ to python in
fftsink & scopesink.  Seems to work fine, and the implementation is
portable across operating systems.

> What I'd really like is to send packets as structured chunks of data,  
> using "connect()" in a graph.

OK.  Are you dealing with fixed length or variable length packets?
If variable length, how are you going to communicate the length?
It is possible to pass fixed length structures around.  We did this
with the ATSC receiver.

I'm interested in your ideas in how to represent variable length
packets within the (modified?) framework.  This would definitely be
useful.  Another (possibly) related things I'd like to do, is to be
able to run a packet through the graph, then, say append 8 zeros, or
some such.  That particular operation is very hard to do with the
current data flow approach.

Others have suggested a "two level scheduler".  The bottom one would
be the existing one (or something close) and the top level one would
know about packet boundaries, etc.  For example, imagine you've
receive an annotated packet from "outside" (typically a MAC). The
annotations could include things like "transmit power", "channel
coding", etc.  You want this high level packet processed in one
logical chunk through the graph.  But then may want to change things
around before or after the next bunch of data.  E.g., trailing
padding.

Still other have suggested a "functional" approach, with
generators/streams wired together and a way to read from / write to
abstract streams.

Another approach would be the "push" / "pull" distinction used in the
"Click Modular Router".

Yet another intersecting issue is the ability to "stall" a piece of
the graph.  The gmsk2_pkt stuff implicitly takes advantage of this,
and only runs the transmit part of the graph when there's data to
sent.  The trick is making sure everything gets flushed out of the
pipeline before you stall.  For example, you don't want a partially
full USB packet (512 bytes / 128 samples) lying around.  The existing
code kludges around the problem.

Good ideas & concerns!  Lets figure out a way to solve it!

Eric