Problem when a 32-bit library is linked instead of a 64-bit library

[Dr. David Kirkby]

Hi all,

I'm not entirely sure if this is a libtool or autoconf issue, though I know the 
two projects are closely related.

I'm having a problem building a package as a 64-bit object on an 64-bit 
OpenSolaris x64 system. OpenSolaris, like Solaris, defaults to building 32-bit 
objects.

1) First a general question. How should one specify to autoconf to build a 
64-bit version of something? (I'm assuming the compiler is GCC or SunStudio, 
both of which use the -m64 option. I'm aware other compilers like IBM's on AIX 
and HP's on HP-UX does not use this option).

I know the following method works in some cases

export CLFAGS=-m64
export LDFLAGS=-m64
./configure

other times (e.g. Python, Singular), one needs to define CC to be "gcc -m64".

The second point relates to a specific failure of a package (libfplll) to build 
on OpenSolaris as a 64-bit binary. What happens is that at the link phase, the 
32-bit version of a library is linked, so it generates an error:

ld: fatal: file /usr/local/gcc-4.4.4-multilib/lib/libstdc++.so: wrong ELF class: 
ELFCLASS32

Instead, it should be trying to link the 64-bit version, which is the file 
/usr/local/gcc-4.4.4-multilib/lib/amd64/libstdc++.so

The following trac ticket on Sage shows this error in more detail, along with 
some debugging options added to libtool.

http://trac.sagemath.org/sage_trac/ticket/7864

Note, that since this ticket was opened, the install directory of gcc was 
changed from

/usr/local/gcc-4.3.4-GNU-assembler-Sun-linker/
to
/usr/local/gcc-4.4.4-multilib

But the error message remains sententially unchanged.

Any ideas how best I can get this to build?

A hack I've found that works, is to add

-L/usr/local/gcc-4.4.4-multilib/lib/amd64 
-R/usr/local/gcc-4.4.4-multilib/lib/amd64

Then the linker finds the libraries first there, so finds the correct 64-bit 
ones, so does not go on to look any further. So the unwanted 32-bit libraries 
are found.

I've previously marked this ticket on the Sage bug tracker as 'needs review', as 
my hack does work, but I've since changed it to 'needs work' as I'm well aware 
my hack is sub-optimal. I'd like to get a more robust solution.

Sorry if this threads seems a bit long.

http://trac.sagemath.org/sage_trac/ticket/7864

I think you can see it looks like gcc is configured to first look in the 64-bit 
libraries, but somehow this is failing to be handled correctly at the link phase.


I'll also try to resolve this on the libtool list (of which I'm not a member), 
as the second part in particular seems more libtool related, thouugh I'm not 
100% sure of this.

I personalty did not write the auotconf/libtool code, and have no knowledge of 
libtool, so I'm a bit in the dark as to know how to proceed.

Dave