RE: Fix for Arg list too long

[Boehne, Robert]

-----Original Message-----
From: Alexandre Oliva [mailto:address@hidden]
Sent: Wednesday, January 31, 2001 2:40 AM
To: Boehne, Robert
Cc: address@hidden
Subject: Re: Fix for Arg list too long

>On Jan 24, 2001, Robert Boehne <address@hidden> wrote:
>
>> #1)  I have to check for the existence of 'wc -c' to check the
>> command line length.
>
>I'd rather not rely upon fixed limits.  I was thinking about doing it
>dynamically.  For example, as the command line grows, test that you
>can still $echo it before committing to the addition of a new
>argument.  But I realize this may be very hard to do right.
>
I agree, I had a bit of trouble with OSF/1 because the system headers
didn't have the right number (or perhaps it was in a different format).
I would prefer to run a test at configure time to determine the
maximum command line length.

>If you really want to use something like `wc -c', make sure you set up
>portable fallbacks, such as expr "X$val" : ".*" and awk's length().
>
I went with 'wc -c' because it was the only thing I could get to work,
but I didn't try awk...  I designed it so that if 'wc -c' was not
available, or the maximum command length hadn't been determined,
or the command line is short enough, then libtool falls back to
the way it currently links.

>Instead of looking for fixed limits in header files, we might just try
>larger and larger commands until we bump onto a limit.  Starting at
>128 and doubling on each iteration until it fails or gets to, say,
>128Kb, is probably reasonable.  The command might be the expr above or
>a non-built-in echo (using the $0 --fallback-echo comes to mind).
>
Alexandre, I'm not familiar with '$0 --fallback-echo', could you
elaborate on this a bit?

>Also bear in mind that the invocation of the char-counter itself may
>fail in case the command line gets too long.
>

I have not had this problem with 'wc -c' but I did have it with `expr $cmd : ".*"`

>> #2)  I am checking the system header files to find the maximum command
>> line length.  Currently I have this in ltconfig.in, but it really
>> only needs to be run once for each platform, not each tag.  Where should
>> it go?
>
>ltconfig is indeed the right place for it (in MLB until we move it
>into libtool.m4, as Gary did in mainline).  So far, we don't have
>infrastructure to do the test only for the main configuration; maybe
>after moving to libtool.m4 we should; that will make it much easier.
>
>> #3) $LD is set to $CXX in most cases for C++ compilers
>
>My opinion is that this is wrong.  We should just bite the bullet and
>use $CXX all over, leaving $LD with the correct value.
>
SGI was one system that would not allow me to use $reload_cmds because
$LD was set to CC. (even with $reload_flag set to -Wl,-r)
I suppose I could read the man pages to see if this can't be
worked around using CC... 

>> A previous design linked objects one at a time
>
>We should probably keep some version of the previous design, at least
>for static libraries.  It's not good to link multiple objects together
>in case they're going to be stored into a static library, because
>linkers often pull whole object files out of static libraries.  We'd
>better add them incrementally (but we can do it many at a time).  We
>should probably have some configuration flag to indicate that
>incremental old_archiving is not possible, and fallback to relinking
>in this case.
>

I'm not sure I understand this, you're saying that some archivers can't
add several object files to an existing archive properly?  This would
certainly be a portability problem I hadn't considered.  Ouch!
  As far as linking one object at a time, it worked, but it increased
the compile times by orders of magnitude.  That wouldn't normally be
a big deal, but in the case where you're linking enough objects to
overflow the command line it becomes unreasonably cumbersome.

  Currently I have made a few assumptions about compilers and linkers,
#1) $old_archive_cmds will allow users to add objects to an existing
    archive.  I'm pretty sure this isn't true, but is there a way
    to add objects to an archive for most supported archivers?

#2) $reload_cmds will take several objects as input and effectively
    concatenate the files.  i.e. linking
       o1.o o2.o o3.o o4.o o5.o
    into a shared library is the same as linking in steps
       $reload_cmds -o newO1.o o1.o o2.o o3.o o4.o
    then linking a shared library with
       newO1.o o5.o
    as input.

For cross-compiles this isn't an issue because it is the linker
on the build system that may run into the arg list limitation.
So perhaps a small C program to check argument lengths wouldn't
be out of order?  Only if 'wc -c' doesn't exist of course.
What language could someone be using with libtool that they
wouldn't have a C or C++ compiler?  What OS'es don't supply 'wc -c'?
I'm not asking to relax these requirements, I just would feel
more at ease working around these limitations if they were
explained to me.

Thanks!

Robert