Re: dlopening installable ltdl modules failing (OS X problem only)

[David Fang]

> > > It is entirely possible that you've found a bug, Mac OS X should work
> > > the same as linux et.al. in this regard. Does the module itself have any
> > > dependencies that possibly are not being found (check with otool -L
> > > mymodule.0.so)? If that is not the case, please send more info (if you
> > > like you can send me the project offline and I'll look into it).
> >
> > Yes, the mymodule.la does have a dependent library, also built.  (otool -L
> > ../../libs/.libs/mymodule.0.so confirms that the dependent library is
> > being referenced in the final installed location via absolute path, where
> > it does not yet exist.)  In fact, if I make another test module *without*
> > any additional LIBADD dependencies in the same place as mymodule.la, the
> > executable is able to lt_dlopen it without errors. So the lt_dlerror
> > message from before is misleading; it's probably finding the first library
> > fine, just not its dependency, and reporting an error on the first
> > library.
>
> The wrapper script for the executable itself should be setting the env
> var DYLD_LIBRARY_PATH to all of the dirs with the uninstalled libraries.
> otool -L will always show the final install location, so that is fine.

Hi,
        So the test case project I posted at
[http://www.csl.cornell.edu/~fang/sw/ltdl_test-0.0.0.tar.bz2] was a bit
too reduced, in that the test executable did not produce a wrapper script
because it was not linked to any built libraries.
        I've revised the case [version 0.0.1] so the executable links to a
built library, which produces a wrapper script.  Now on my darwin build,
the executable's wrapper script contains:
DYLD_LIBRARY_PATH="/path/to/ltdl_test/src/.libs:$DYLD_LIBRARY_PATH" as
expected.  Herein lies the problem.  Any libtool libraries that are built
*outside* of src/ (where the executable is built) are not automatically
added to this library search path.  (How can it possible know without
additional guidance?)  This might explain why we see breakage as soon as
the module being dlopened depends on another built library (that happens
to not be in the automatic DYLD_LIBRARY_PATH).
        Now if I run (with environment)
"DYLD_LIBRARY_PATH=/path/to/module/depend/libs/.libs make check" the tests
all pass.  This seems to solve the problem.  I was only mildly surprised
that this is necessary on darwin vs. linux or freebsd, all using
libtool-1.5.24.

> check that the wrapper script is in fact adding all of the required dirs
> to DYLD_LIBRARY_PATH, also check dyld(1) for more potentialy useful
> environment variables (DYLD_PRINT_LIBRARIES is useful).

        So I think the lesson is this: when testing against built modules
with library dependencies that do not lie in the same directory as the
built executable, add to the test environment:
$(LTDL_SHLIBPATH_VAR)=path/to/module-deps/.libs.  Affected systems:
Darwin.
        Unfortunately, LTDL_SHLIBPATH_VAR is only emitted as an AC_DEFINE
in config.h, I'd now like it to be AC_SUBST'd to appear in Makefiles (as
well as LTDL_OBJDIR):
[configure.ac excerpt]

LTDL_SHLIBPATH_VAR=$libltdl_cv_shlibpath_var
AC_SUBST(LTDL_SHLIBPATH_VAR)
LTDL_OBJDIR=$libltdl_cv_objdir
AC_SUBST(LTDL_OBJDIR)

With the "workaround" of appending additional paths to the
libary search path, everything works "as expected".

> But wait a second, talking about DYLD environment variables made me
> remember the security update:
> http://docs.info.apple.com/article.html?artnum=305509
>
> Does `sudo touch /var/db/.AllowDYLDEnvironmentVariables' fix it for you?

I tried this (on darwin8), and nothing changed.

I've reposted the test project at:
http://www.csl.cornell.edu/~fang/sw/ltdl_test-0.0.1.tar.bz2

Revision:
Test executable is now built as a wrapper script.
configure now substitutes LTDL_SHLIBPATH_VAR and LTDL_OBJDIR.
Two new tests in each dir (lib/ test/) prefix the two existing failing
test (darwin only) with an augmented library search path environment,
which now pass.

Can you (or anyone) reproduce my findings with test project 0.0.1?
Am I Doing The Right Thing (TM)?


Fang