[Discuss-gnuradio] libtool/linker/dynamic loader stuff

[Eric Blossom]

It seems like it's time to fully revist the interlibrary dependency
issue, and the related issue of reliably and robustly testing with
build tree libs vs install tree libs.

My priority order for this working is:

  Free systems are first priority:

  * GNU/Linux must work as expected using unmolested libtool.
    Ubuntu may be broken, and if it is, we need to fully understand
    why and how it's broken.  If it makes sense, we should file a bug
    upstream with good explanation for the behavior that we expect.
    I.e., testing with uninstalled libs.

  * NetBSD must work as expected

  * OS/X is next (at least their kernel is free)

  * MinGW/Cygwin/Windows:  lowest priority.  If we have to break this
    to fix the others, it's not the end of the world.  I'd prefer a
    solution that works reliably for all, but if it comes down to triage,
    this one dies first.


What "working" means:

We are able to test our apps and Python code in the build tree in a
way that is _completely_ independent of whether or not there is an
earlier installation of our code or not.  I've seen some flaky
behavior on some platforms where it appears that we get different
behavior depending on whether there's already something in the install
path.  This usually manifests itself as "undefined foo..." where foo
is the mangled name of a new function or method that is in fact
contained in the build tree libs, but isn't contained in the installed
libs.

When make check is run, the installed libs, if any, MUST NOT BE
REFERENCED to resolve any symbols.

Likewise, when installed, our libraries should only resolve symbols
using the installed libraries.

I believe that libtool will do this out of the box.  It normally
causes stuff to be linked twice: once at compile time, and once again
at install time.  However, for this to work, we need to do things "the
libtool way".  I'm not sure we're fully compliant.

I'm looking for someone who's willing to sort this out and to test on
all platforms.  To succeed in this venture the person ought to have
(or be willing to develop) a solid understanding of how dynamic
linking works in general, how libtool works, and what ld flags such as
-rpath, -L, and -l do.  Understanding how python finds and loads
native modules is essential too.  OS-specific knowledge of dynamic
linking and shared libs on all the platforms would probably come in
handy ;)

Under Linux, see man ld, man ld-linux.so, man ldd, man ldconfig
Under OS/X, don't confuse glibtool with libtool ;)

http://www.gnu.org/software/libtool/
http://www.gnu.org/software/libtool/manual.html
(You can also build the pdf doc if you download the libtool src)

You'll also want to look at the automake docs and see how it interacts
with libtool:

http://www.gnu.org/software/automake/
http://sources.redhat.com/automake/automake.pdf

http://www.gnu.org/software/libtool/manual.html#Using-Automake


Relevant environment variables include:

  LD_LIBRARY_PATH, PYTHONPATH, DYLD_LIBRARY_PATH (os/x)


In all of this discussion, please assume libtool 1.5.22.  It's the
latest stable version (18 December 2005).


Any volunteers?  Fame and glory awaits!


(You don't need to be an existing committer.  If you're willing to take
this on, I'll set you up with the access you need to do the work.)

I'm sure we can find a way to provide access to all the relevant platforms.


Thanks!
Eric