Re: INSTALL matters

[Bruno Haible]

Hi Neil,

Neil Jerram wrote in reply to
<http://lists.gnu.org/archive/html/bug-guile/2008-05/msg00012.html>:
> This issue, and the related (in
> my mind) one about having multiple copies of a library installed in
> different places

Having multiple copies of a package installed in different places is
the most normal thing of the world. That's what --prefix is for.

> First, a couple of assumptions: as far as I understand,
> 
> - in general, there is no portable way of telling the compiler and
>   linker to look in one set of prefixes for library A, and in a
>   different set of prefixes for library B

There is. It consists in passing absolute file names for .h and .a or .so
files to the linker. If the input you have is a _set_ of of prefixes,
this means you need to search for the library in the same way as the
compiler or linker would do, and then pass the absolute file names to the
compiler or linker.

It is a bit far-fetched, but the mentioned macros in gnulib actually do this.

Without extra efforts, however, your statement is correct.

>   specifically, therefore, if you have multiple versions of libraries
>   installed in /usr/lib and /usr/local/lib (or /opt/lib, whatever...),
>   there's no way of reliably getting the /usr/lib version of library A
>   and the /usr/local/lib version of library B

Correct. Even if you pass absolute filenames and '-rpath', on some systems,
the information written into the executable by the linker does not contain
sufficient information for choosing library A from /usr/lib (and not
/usr/local/lib) while at the same time choosing library B from /usr/local/lib
(and not /usr/lib).

> - worse (at least with the GNU toolchain), if you have multiple
>   versions of library A in /usr/lib and /usr/local/lib, you can't get
>   a consistent build at all, because gcc prefers /usr/include when
>   looking for headers, but ld and ld.so prefer /usr/local/lib when
>   linking.  (Or perhaps the other way round.

This is a problem on some BSD systems, IIRC. Also one specific GCC release
had this problem on Linux systems, a couple of years ago, and it was fixed
in the next GCC release. So I've come to view this as an inconsistency in
the GCC installation. The workaround to this situation is to never use
/usr/local, but rather pass a different --prefix always.

> > I'm trying to install guile-1.8.5 on MacOS X 10.5. I've already installed
> > gmp-4.2.2 with the same --prefix option as I use for guile,
> 
> This suggests a possibility, of automatically adding $prefix/include
> to CPPFLAGS and $prefix/lib to LDFLAGS.

Yes. If I have installed gmp with the same --prefix option, then this
is needed and desired. Autoconf does not do it automatically.

> > guile's README has two paragraphs about this topic. Both are less than
> > helpful:
> >
> >   - The first paragraph says that the installer should consider -I options,
> >     but does not say how to pass them (CFLAGS? wrong! CPPFLAGS!), and 
> > moreover
> >     does not even mention that LDFLAGS need to be set as well (for -L and
> >     rpath related options).
> 
> Completely agree that this is not helpful enough.  I'll propose an
> update once we're agreed on the overall picture.

Thanks.

> >   - The second paragraph recommends to rebuild GCC to match --prefix. This
> >     is just gross. It takes a novice user half a dozen attempts to find out
> >     the right set of configure options for building gcc, and then the build
> >     itself takes 10 hours and requires 500 MB of swap space.
> 
> The README doesn't recommend this; it says that it is a "convenient"
> option.

We must have different interpretations of the term "convenient" :-)

> In some specific contexts - e.g. an admin setting up a box with all
> 3rd party software under /opt - I think that this would indeed be a
> convenient option.

I don't think so. The users of this admin will wonder why their programs
behave differently when built with GCC than when built with the vendor's
compiler. (The reason would be that GCC links with different versions of
thirs-party libraries.)

> I would prefer to document really clearly how to use CPPFLAGS and
> LDFLAGS, rather than adding these options.

That would be fine with me as well. The installer can be expected to
look into the README and INSTALL files and apply the instructions found
there.

> I don't like these options because they seem bogus to me.  Providing
> `--with-libreadline-prefix' makes it sound like the build and runtime
> will look specifically (and only) in that prefix when building/linking
> readline, and in the default places for other libraries.  And that is
> not true

It is correct that the option does not mean "hey configure, please
look for libreadline *exactly* here". It means "hey configure, I have
installed libreadline here".

Yes there are weird cases which cannot be supported, like the case
with libraries A and B in both /usr and /usr/local, which you mentioned
before. But the majority of the cases is not so weird, and in these
cases the --with-...-prefix options are helpful.

> Using CPPFLAGS and LDFLAGS is better, in my view, because it
> corresponds to what the toolchain can actually do - i.e. search in one
> global set of places for all headers, and in one global set of places
> for all libraries.

I find a configure option better, because when the user installs a
package, he's not thinking at the level of linker details, but rather
"I have installed gmp here. Now how do I tell mpfr or guile about it?"

> Therefore I would like to document the proper use of CPPFLAGS and
> LDFLAGS, explain when these are needed, and also explain the problem
> with having multiple library versions installed.

Fine with me, if following this documentation yields a working build.

> Now coming back to the point above about automatically adding
> $prefix/include to CPPFLAGS and $prefix/lib to LDFLAGS...  It's an
> interesting idea, but I'm sure that it would cause unwanted build
> behaviour for some people, and it should be trivial (given proper
> documentation) for people who _do_ want this to set their CPPFLAGS and
> LDFLAGS; so on balance I don't think we should do this.

Before I came to use --with-...-prefix options, I used CPPFLAGS and
LDFLAGS like you say. And quite often, the build failed because
  - I had not been aware of all dependencies before building the package, or
  - I had used LDFLAGS="-L/opt/gnu", rather than LDFLAGS="-L/opt/gnu/lib",
    (Simple typo/thinko, but frustrating anyway.)
  - I had used LDFLAGS="-L/opt/gnu/lib" but the build needs the 64-bit libraries
    so I should have used LDFLAGS="-L/opt/gnu/lib64".

The --with-...-prefix options do reduce frustration in these cases.

> > As an installer, I'm not going to spend time trying to see which LDFLAGS I 
> > need
> > to set for a package to recognize the libraries that I have installed. I 
> > expect
> > the package to do this by itself.
> 
> I don't get this last paragraph.  You have to add a path either to
> --with-xxx-prefix or to CPPFLAGS and LDFLAGS, so what difference does
> it make?

The difference that I meant is when I use the same --prefix for a couple of
related packages. Suppose my goal is to install 'guile'. I learn that it needs
'gmp', so I download and install that, with the same expected prefix. Then I
install guile with the same prefix and it refuses to acknowledge the previously
installed packages, that it frustrating.

I know that some of these problem come from the fact that GNU does not have
a package registry or package dependency tracking tool. That's actually good
because it gives much more freedom than 'rpm' or 'dpkg' do. But when a package
considers a dependency to be absent when it's actually present where you would
expect it, this needs correction IMO.

Bruno