Re: [gpsd-dev] RPATH tangle

[Eric S. Raymond]

Miroslav, there's some stuff down near the end of this you'll want to
pay particular attention to. Same goes for Mike Frysinger and other 
packagers.

I'm putting a hold on 3.11 release until we resolve this.  Otherwise, 
all is in readiness.

Hal Murray <address@hidden>:
> First, a request/suggestion:
> 
> scons prints out things like:
>   Altered configuration variables:
>   bluez = False (default True): BlueZ support for Bluetooth devices
> They usually scroll way off screen so nobody sees them unless they know 
> enough to go looking for them.
> 
> How about saving important messages and repeating them at the end of a run 
> where they will be easy to see?

Mainly because if I do that I anticipate endless arguments over which
messages are "important", and some of the ones people will want repeated
are difficult or impossible to capture.

Running scons a second time (when they won't scroll offscreen, because 
the main build targets have been made) is an easy workaround.

> address@hidden said:
> > The mess around chrpath now seems resolved.  Judicious deployment of
> > LD_LIBRARY_PATH settings in regress-driver and the scons library has made
> > embedding $PWD in RPATH unnecessary for people running the standard
> > regression tests; only ad-hoc tests (e.g. running gpsd or gpsmon by hand)
> > are a problem now.  Accordingly, I have changed the configuration default to
> > chrpath=no and removed it from the dependency 
> 
> I think defaulting chrpath=no is evil.  I could probably learn to live with 
> it, but I'm likely to make stupid errors when trying to test things.
> 
> I think it should "default" to yes if chrpath is available, and if not 
> available or you specified chrpath=no, scons should print a warning message 
> reminding you that ./gpsxxx needs LD_LIBRARY_PATH or such.  The details could 
> be in a README type file.

They already are; build.txt has a section about this.  As for changing
the default, that might be viable.  I'm going to try to extend your
case analysis and think through all the possibilities.

Here's a factor you may be unaware of: it's been made clear to me
since I originally introduced it into the build that distribution
packagers really dislike chrpath.  It's an extra dependency, it's not
very well maintained, and it has caused cross-build problems (notably
on OpenWRT).  Not having it be a required build component is good.

> Maybe the concept of things defaulting to yes or no is not good enough.
> 
> Maybe we should add something like test=yes so the no-chrpath case can build 
> with the local dir in RPATH.

See the case analysis below.

> There is a corner of scons that I don't understand.  Some (maybe all?) 
> configuration variables seem to be sticky.  That may make it much easier for 
> me to work with a default of chrpath=no.  Once I set it to yes, it will stay 
> that way while I run scons from the command line.

All configuration variables are sticky.  Once you set
chrpath={yes|no}, it will stay that way until you unset it or remove the
configuration stores at .sc*.

> How many use cases are there?
>   Packager:
>     needs to work without chrpath
>     scons check should work both with and without chrpath
>   Developer:
>     Is it OK to assume chrpath is available?
>     ./gpsxxx needs to use local libraries.
>     scons check should work

Add another case, the power user.  Not a dev, but able to build from
the repository, and often wants to in order to use or test features
that haven't made it into a release yet. We have a couple of these
on the user and dev lists.

Power user:
  Needs scons check to work.
  Could install chrpath, but would rather not have the complexity overhead.
  Unlikely to run ad-hoc tests without installing the libraries as root.

In the packager case we're already good without chrpath.

In the developer case I think it is OK to assume that chrpath is
available.  It's not a lot of effort to install. But because the build
needs to work for packagers and power users, we can't have it error
out if chrpath is not present.

Since the controlled regression-test environment within regress-driver
and the scons recipe now has LD_LIBRARY_PATH set, whether scons check
works or not is now independent of the chrpath setting.

The tests can, however, have other problems - notably, you've
discovered that /usr/local/lib may be searched before $PWD.
I'll go into that problem a bit later.

Possibilities:

chrpath defaults to no: Packagers happy. Power users OK.  Developers
trip over this when running ad-hoc tests and have to install chrpath and 
set chrpath=yes.

chrpath defaults to yes: Absent chrpath aborts the build, must recover 
by setting chrpath=no.  Packagers and power mildly unhappy with the
extra configuration complexity, developers shrug and install chrpath 
instead.

chrpath configuration defaults according to whether chrpath is
installed: Packagers now extremely unhappy, because whether an
incautious build introduces a security hole is now dependent on the
installation state of a third-party package not obviously related to
GPSD. Power user were OK with chrpath=no, so no gain for them.  Developers
still trip over ad-hoc tests until chrpath is intalled.

The only people who gain in the third case are developers who already
have chrpath installed.  That gain doesn't seem to me to be worth the
more serious unhappiness it would cause packagers.

So chrpath=no seems like the right thing to me, though  I wouldn't have
said that before we could guarantee regression tests would still work.

> I think the current RPATH stuff is a bit confused.  With chrpath=yes, I get:
>   ./gpsmon: RPATH=/home/murray/gpsd/gpsd2://usr/local/lib
> I think the second part, /usr/local/lib is wrong.  I don't want it using the 
> installed libraries.
>
> PS:  Current RPATH gets set to "//usr/local/lib" (extra /, no big deal, just 
> ugly)

I've looked unto this, and it turns out this is why you were getting
/usr/local/lib searched first.  It appears the link search order is:
first /lib/ and /usr/lib.  Then other locations on the system default
load path set by ld.so, which may or may not include /usr/local/lib.
Then directories that were in RPATH at the time link stubs were
generated. Finally, at runtime, directories in LD_LIBRARY_PATH.

Currently these are the only places we mess with RPATH: 

if env["shared"] and env["libdir"] not in {"/lib", "/usr/lib"}:
    env.Prepend(RPATH=[installdir('libdir')])

if not (env.GetOption("clean") or env.GetOption("help")):
    if env["shared"] and env["chrpath"]:
        if WhereIs('chrpath'):
            # Tell generated binaries to look in the current directory
            # for shared libraries so we can run ad-hoc tests without
            # hassle (the regression tests *don't* need this as
            # they're run in a controlled environment where we can set
            # LD_LIBRARY_PATH). Should be handled sanely by scons on
            # all systems.  Not good to use '.' or a relative path
            # here; it's a security risk.  At install time we use
            # chrpath to edit this out of RPATH.
            env.Prepend(RPATH=[os.path.realpath(os.curdir)])
        else:
            print "chrpath is not available; please build with chrpath=no."

The first piece of code is what produces your wrong behavior.  The
obvious fix would be to add "/usr/local/lib" to the exclusion list -
but that has a problem. It will lead to obscure link failures if someone
does a prefix-/usr/local defaulted install and /usr/local/lib doesn't
happen to be on the system load path.

What we really want to do here is ask the system what the default load path
is and not set RPATH if libdir is already on it.  I've been looking for
a way to get this informastion, but there doesn't seem to be an option of
ld, ld.so, or ldconfig that produces it.
-- 
                <a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>