Re: libtool and mkoctfile

[Thomas Weber]

Hi, 

I'm picking Jaroslav's message for answering, but I hope I've added
everybody currently involved into the CC. 

On Wed, Nov 04, 2009 at 10:52:56AM +0100, Jaroslav Hajek wrote:
> On Wed, Nov 4, 2009 at 9:50 AM, Benjamin Lindner <address@hidden> wrote:
> >
> >> I would like to make the mkoctfile script use libtool to create .oct
> >> files, the same as we will soon be using in the Makefiles that build
> >> the .oct files that are part of Octave.  I think I have most of the
> >> details worked out, but one problem remains.  Libtool is a Unix shell
> >> script, so we will need to do something for Windows systems.  Is it
> >> reasonable for Octave to require a Unix command shell and the
> >> associated commands that libtool requires for building .oct files?
> >
> > Frankly, no. I'd like to return the question by asking is it *really* 
> > necessary to rely on a unix-sh-interpreter?
> > I see mkoctfile as an integral part of Octave an if mkoctfile requires a 
> > unix-layer to run, then, well, you won't have a native windows binary any 
> > more.
> > And I wouldn't see the point in providing mingw binaries with the 
> > additional necessity of a unix shell, when there's cygwin available which 
> > does exactly all this, and I guess much better.
> > And what about msvc? There are no 'official' binaries, but octave can still 
> > be built with ms compilers I guess and run as native binary.
> >
> > What about the future development, what about e.g. x64 binaries?
> > I'm not sure about unix-shells-for-win64-which-are-not-cygwin. Possibly 
> > cygwin is the better choice as unix layer - and then I'll end up again at 
> > the question, why provide mingw32/64 along with a unix shell interpreter 
> > when there is cygwin?
> >
> > Aside from the question of unix-sh-interpreter yes or no, I disagree to 
> > using libtool, at least for windows platform. Simply because libtool does 
> > not work properly for windows.
> > There are problems creating shared libs, there are problems with compiler 
> > flags, there are problems with libtool dropping libraries from the command 
> > line at own gusto, there are problems with shared libstdc++.
> > So I would most probably end up packaging a patched (i.e. hacked) version 
> > of libtool which I honestly could not support because i simply don't 
> > understand it myself.
> > Using libtool for building octave is using it in a controlled, and kind of 
> > encapsuled environment where one knows the odds and ends and where you have 
> > a bunch of workaround tricks in your pocket.
> > But distributing it and depending on it in open space, I believe will end 
> > up with many reports of "mkoctfile does not work for me" and us saying, 
> > "yes, we know, this is libtool".
> >
> >> If not, then I suppose that when Octave is built, we could try to
> >> extract from libtool the commands that it would run, and embed those
> >> in the mkoctfile script/program, though I'm not sure how to do that in
> >> a reliable way.  If we do go this route, then I would like to consider
> >> again eliminating the mkoctfile script and keeping only the C++
> >> program.  Having both seems like asking for trouble as the two are
> >> likely to get out of sync.
> >>
> >> It seems easier to me to just package a Unix shelland the necessary
> >> commands for Windows systems, then eliminate the C++ version of
> >> mkoctfile.
> >
> > I'd like to stick to the C++ version of mkoctfile and I would have no 
> > problem with having a (windows) C++ version alongside the (common) shell 
> > version. You are right about having two versions and them possibly 
> > diverging, but how much development in the future do you honstly see in 
> > mkoctfile? How much new features do you plan to add to mkoctfile? I think 
> > that the additional effort of keeping the C++ version in pace with the 
> > shell version small compared to forcing a shell/libtool version for windows 
> > and maintaining it.
> > I'd rather be one step behind with the C++ version than requiring a unix 
> > shell for it.
> >
> > To me, the C++ interface is octave's biggest bonus aside of it being free 
> > software. And to be able to use the C++ interface natively with a free 
> > compiler available on windows, I wouldn't want to miss that.
> >
> >> If you object to including a Unix shell, why?
> >
> > I'd object to having a unix shell as runtime requirement for octave. It'd 
> > end up in a windows binary depending on a unix layer, and that's cygwin.
> >
> >> Suppose that libtool
> >> were written in Perl.  Would you also object to including perl in the
> >> distribution so that Octave could run the libtool script?
> >
> > Possibly. I have no experience with perl distributions for windows, so I 
> > cannot estimate the amount of effort it would require to include it.
> > I'd probably stick to a C++ version of mkoctfile anyway.
> >
> > This is also getting a bit philosophical.
> >
> > At the current stage, windows binaries of octave do not require any 
> > privileges for installing a default user (who is not administrator) on 
> > windows doesn't have.
> > It even doesn't require being "installed" at all, you can just copy it and 
> > run it off a usb stick, if you like. It doesn't need to fiddle with windows 
> > registry and it can be cleanly uninstalled or removed by simply deleting 
> > the directory tree it was installed to.
> >
> > Now beginning to add dependencies that don't follow this philosophy, may 
> > mean that octave is no longer portable, and octave must strictly be 
> > installed with administrative privileges. And this is IMO the wrong way to 
> > go.
> >
> >> Octave
> >> already includes a perl function.  Do we distribute a copy of Perl
> >> with Octave on Windows systems so that the perl function can work?
> >
> > No, perl's not included. Now I'm curious. Which octave function is that?
> >
> > benjamin
> > --
> 
> The only perl script is octave/mk-opts.pl, and it is only used to
> recreate the option handlers DASPK-opts and similar. I think that like
> awk, bison, flex and gperf, perl is only needed to build from plain
> repo, but not from the distribution tarball.
> 
> If libtool was only required for generating distrib tarballs, it would
> make little difference, but making it a requirement for mkoctfile
> would mean a significant change. Not just for Windows; unless I'm
> missing something, it would mean that dependency on libtool would be
> required for Octave packages in GNU/Linux distros as well.

Okay, from my viewpoint of Debian maintainer: I would jump for joy when
Octave uses libtool.

In fact, two or three years ago I looked into adding libtool to Octave,
but I gave up when I realized that Octave didn't use automake (which is
not strictly necessary, but the libtool manual only documents the
automake use case).

libtool is not some esoteric tool, but sort of standard for libraries in
the Unix-like world. Using such a tool means you can ask on normal
distribution lists and there will be literally hundreds of people with
experience on this topic.  Whereas currently it means asking on the
Octave lists and hoping that the few people here have an idea. Which, I
might add, usually boils down to John in use cases like Solaris. Thus,
I'd actually expect people to give him free reign in his decision or
stand up and help the next time a question about Solaris comes up.

Now, some things that came up in different mails in the thread, by
different people:

| For windows it simply doesn't make any sense to use it, just to get
| source code compiled and linked. Creating an oct file is a one-liner
| with the gnu compiler, and using a nearly-300K sized shell script to do
| the job seems - well - strange?

Yes, it takes one line. And it takes one line on Linux. And one line on
Solaris. And one line on HP-UX. 
Unfortunately, these lines might differ on all of these systems. They
might differ when using a different compiler. They might differ when
using a different toolchain. Does anybody on this list know how to
compile a shared library on VMS? I don't. But libtool does.

http://www.fortran-2000.com/ArnaudRecipes/sharedlib.html
to get an idea of the fiddling required.

| If libtool was only required for generating distrib tarballs, it would
| make little difference, but making it a requirement for mkoctfile
| would mean a significant change. Not just for Windows; unless I'm
| missing something, it would mean that dependency on libtool would be
| required for Octave packages in GNU/Linux distros as well.

I already had to patch mkoctfile to make it work reliable in Debian (it
hardcodes paths when building Octave, which makes up for problems when
the compiler is updated). So, no, mkoctfile is not a simple
one-size-fits-all solution.

Is libtool better? I don't know. But there are certainly more people
with knowledge about libtool than there are with knowledge about
mkoctfile and compiler flags.

| With all due respect, but at this point I'd like to re-raise the
| question, why are you trying to switch to libtool in the first place?
| What's so complex about compiling a source file and linking a library
| supposing for simplicity one is using the gnu compiler/linker suite?
| (Does libtool even support different compilers?)

I can only wonder that people argue about portability in one sentence,
yet don't care about totally different compilers and platforms in the
other sentences. You want portability? Use proven tools. And yes,
libtool is complex and actually pretty slow when run. The fact that it's
still around after all these years should however be a sign that
somehow, it's deemed useful.

        Thomas