Hello,
good to see so much feedback to this proposal. This feature is
clearly something that we all want to see become reality!
I have discussed this idea with Kartik beforehand offline and I am
glad to see all of you have the same issues/objections.
While bokeh (and plotly) were mentioned in the proposal only
because the idea was inspired from these libraries, they maybe not
the be a natural fit for us after all - at least on the server
side. Maybe, at this point, it would be useful to specify what we
actually want out this and then look at what Bokeh and others
provide us to get there. Even if that means, that this module will
not have all features after 3 months of coding that we want long
term.
My rough idea of this is
- a server that runs an event-loop to push push data to clients
and receives/handles events from them
- a single websocket connection per client which the data (for all
plots) is multiplexed using messages
- each sink block holds a number plot data buffers ("O"s may
happen) that the event loop async pulls from and sends to clients
- metadata is a separate message?
As far as I can tell this is pretty much what Bokeh does, except
that we want a more direct path from our blocks to the websocket
stream. I'm not sure if the Bokeh server has an lower-level API
that we could we. The other thing I am unsure about is the flow of
data, that Marcus mentioned. Ideally we could have something with
the event loop in Python that allowed us to stream data stored in
buffer object (build in at the C level).
Kartik, I think you should include a more detailed description on
how you plan to control the data flow and maybe, expand on the
planned options of the placement of widgets.
Sebastian
On 03/21/2017 02:09 PM, Marcus Müller wrote:
Hi Kartik,
thanks for the feedback!
so, I took the time and tried to read up a bit on what Bokeh
does, how it's partitioned. I'd like to describe this here as
shortly as possible, mostly for my own understanding and to
foster more discussion with others that can't find the time, and
would kindly ask you to confirm this is your understanding, too:
- Bokeh is Python library that produces plots
- It's plotting frontend is HTML & _javascript_
- This enables interactive plotting (ie. you can zoom in, pan
etc without updating the dataset visualized)
- The frontend _javascript_ is a library (BokehJS) used in a
"standard" piece of code that you feed two things:
- The ID of the HTML <div> tag to be filled with the
graph
- The data and plot settings to be visualized as JSON
- Bokeh (the python library) comes with a Python-implemented
server that you're planning to use
Now the things I'm not sure about:
- said server has a REST API, i.e. is polled from the
client only? Or can there be pushing data from server ->
client? What is the model you'd prefer? Your proposal says
"stream to the clients", but I can't find that in Bokeh (but
then again: no experience on interactive Web dev on my side at
all, so this might be really easy for you, if possible).
- I've no idea how to talk GR data -> bokeh-server. Maybe
you could elaborate on how you do that?
- the JSON is relatively verbose, and contains basically all
imaginable settings for a plot. However, I presume a data
update from the bokeh-server would only consist of the model
data, which, for a 5-point line graph would consist of (this
is pretty-printed, Bokeh omits all the whitespace/line
breaks):
{
"type" : "ColumnDataSource",
"id" : "ce58de4d-0ef2-43cf-b15a-23d431781c1a",
"attributes" : {
"callback" : null,
"column_names" : [
"y",
"x"
],
"data" : {
"x" : [
1,
2,
3,
4,
5
],
"y" : [
6,
7,
2,
4,
5
]
}
}
},
- Hence, if performance of the python server is doubted, or
integration into GR is complicated, you could also just write
a C++ block that takes numerical values from GNU Radio and
serves them via Websocket (how does one do that?), and point
BokehJS at that instead? (example from [0], found the
embed_items declaration in [1])
- However, when running [2] and changing the data source, I
see all the plot JSON being transferred, the ColumnDataSource
object just being about half of the 12 kB response object
(amongst dozens of requests made to react to this change of
model, but that might be an effect of this being a
feature-rich plot).
- Can you please explain the user work flow? I would have
imagined that the users design their "plot layout" either
- implicitly (by just using your sinks in their flow graph),
so that gr-bokehgui builds a HTML page containing the plots
they want on its own, or
- explicitly by designing an HTML page with <div>
placeholders on their own and tell gr-bokehgui "hey, use
that; the plot <div> IDs are [list of IDs]",
- but I'm not sure you'd support the second method, since you
mention the ability to add labels and such, which would be
relatively redundant with that.
Best regards,
Marcus
[0] https://demo.bokehplots.com/apps/movies
[1]
https://github.com/bokeh/bokeh/blob/0.12.4/bokehjs/src/coffee/embed.coffee#L161
[2] https://demo.bokehplots.com/apps/crossfilter
On 21.03.2017 12:07, Kartik Patel
wrote:
Dear Ben and Bastian,
Thank you very much for the feedback.
@Ben: Thank you for this error. I have not tried to
dig deep in this error because it was just a
prototype.
@Bastian: You got the server architecture correct.
I am planning to do exactly same.
Coming to queries about using Bokeh:
I agree that we are all-in to Bokeh. I also agree
that it is good to have different modules for frontend
and backend. But in that case we will be simply
replicating Bokeh's backend (with much less features)
and Bokeh's frontend (independently this time). I
believe it will be like re-inventing the wheel.
Also, the features like simultaneous change in
parameters in all connected documents are necessary
when working with the hardware or even during
simulation. Implementing this feature and some more
like these will not be easy. Also, considering that
Bokeh library includes all these in itself, I think
it's implementation will be more efficient than what I
will make. (This last statement is an assumption. Not
sure how valid.)
So, I think now only issue is unreliability of the
library. In other terms, if somehow in future Bokeh
become obsolete, then this entire module will die. In
order to become less dependent on this, we have to
make a independent manual python based server and
Plotly based frontend as suggested.
It means we have a trade-off between the scope of
the project and reliability of Bokeh. I am not sure
how good it is to develop something which already
exists because of the possibility that the library
will be obsolete one day. We can discuss on this.
Only reason, I am resistant to the idea of not
using the Bokeh server is that the existing features
and scope of the project will be reduced. Since, all
the plots and widgets are necessary in order to make
the module usable, limiting the plots will not be good
enough. Although I am open to contributions on this
module even after GSoC, this is not valid statement
for the proposal in GSoC.
I am fully aware with the probable issues of having
the external library as core dependency. I can also
imagine the module having independent frontend and
backend. But is it worth rewriting the code that exist
in Bokeh?
If we still decide to go for independent backend
and frontend, I will need one weekend to write full
proposal for that. So, it would be good if we can
conclude the discussion before this Friday.
Thank you very much for the suggestions.
|
_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
|