[Discuss-gnuradio] GNU Radio documentation

[Eric Blossom]

Hi everyone!

I was hoping to get a mailing list set up for us on gnu.org, but
that's taking much longer than expected.  In the mean while, I thought
we could sketch out a documentation plan.

The current state of affairs is that pretty much all of the C++ code
has inline documentation that's extracted by doxygen to generate the
C++ documentation of the classes and functions.  The doxygen output
looks like this:  http://www.gnu.org/software/gnuradio/doc
I typically go in through the "Class Hierarchy" link on the main page.

This is a good start, however some of the higher level portions of GNU
Radio, including all the flow graph stuff and examples are implemented
in Python and aren't visible to doxygen.

In my ideal universe, I'd have at least two kinds of documentation.
The first is the kind that can be directly extracted from code.  I
like this because the documentation source is in the code and is
likely to be kept up to date.  In addition the tools understand the
programming language and can generate dependency graphs, inheritance
diagrams, etc without human intervention.  Currently the documentation
is in the C++ .h files.  All comments that start with /*! ... */ or
//! are specially parsed by doxygen.

The second kind of documentation is the old fashioned kind, written by
humans, that serves to introduce the territory, explain how things
work, etc.  We are in serious need of this type.  For sake of
discussion let's call these two types "machine extracted" and "hand
written".


Tools for the Hand Written kind:
--------------------------------

After spending yesterday surveying the current systems for producing
documentation, I've come to the conclusion that DocBook XML is the
current winner.  It's future proof and being used by GNOME, KDE,
the Linux Documentation Project, Sun, FreeBSD, Rational, and Apple.

For those of you not familiar with DocBook it's fundamentally an XML
based structure markup language.  You don't specify how anything is
supposed to look; rather, you specify what the things in your document
are, and the rendering is handled as a separate step.  DocBook can be
easily rendered into HTML, PostScript and PDF.

I don't want to scare anybody off with this DocBook stuff.  If you
just want to write straight ASCII, that's fine by me, somebody will
convert it into DocBook for you.  If you're willing to explore
DocBook, two good introductory resources are Eric Raymond's "DocBook
Demystification HOWTO"
http://www.tldp.org/HOWTO/DocBook-Demystification-HOWTO/index.html 
and Michael Smith's "Take My Advice: Don't Learn XML"
http://xml.oreilly.com/news/dontlearn_0701.html


Over the next few days, I'll create yet another CVS module, probably
called "docs", that will have Makefiles and at least skeletons for a
few documents.

 * How to Write a Signal Processing Block
 * Getting Started with GNU Radio and the USRP

I welcome other proposals.  Perhaps a walk through particular examples?

We should also update the two Linux Journal articles and add them to
our documentation pool. The code for the first one needs a bit of
massaging to work under 2.x (any volunteers?).  The second one should
be fine as is.


Tools for the Machine Extracted kind:
-------------------------------------

I'd like two sets of machine extracted documentation.  First, the
existing C++ doxygen output, and second, a python focused version that
includes all the main python interfaces, plus a python view of the
relevant C++ stuff.

Epydoc (http://epydoc.sourceforge.net) is a doxygen like system for
python.  It extracts in-line docs from the python code and combines
that with information that it derives by using python's introspection
features.

We use SWIG, the Simple Wrapper Interface Generator
(http://www.swig.org), to automatically generate the glue that
connects our C++ classes and functions into python.  There is a SWIG
interface file -- a .i file -- for each piece of functionality that we
export to Python.  A .i file is basically a pared down version of the
C++ .h file.  Sometimes there's a bit of extra stuff, but mostly we
leave stuff out that's not of importance to the python interface.

For somebody looking for a programming challenge, it would be great to
come up with a way to generate epydoc input from the .h files (source
of the C++ in-line docs), the .i files (which parts of the .h file we
actually exported to python) and some heuristics regarding type
mapping from C++ to python (e.g., vector<float> => python sequence).

It's probably worth figuring out how the wxPython people are
generating their new epydoc based documentation.
http://www.wxpython.org/docs/api/ 
They're basically in the same boat we are: a big C++ library with SWIG
glue connecting it to python.

That's it for now!

Eric