[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [MLB] Fix for arg list too long (another attempt)
From: |
Robert Boehne |
Subject: |
Re: [MLB] Fix for arg list too long (another attempt) |
Date: |
Mon, 19 Feb 2001 20:03:35 -0600 |
Here is another attempt at an incrementally linking libtool.
I would like anyone so inclined to look at the ltmain.in patch
and point out any portability problems, and please make
suggestions if you have them! ;)
Alexandre Oliva wrote:
>
> On Feb 15, 2001, Robert Boehne <address@hidden> wrote:
>
> > Alexandre Oliva wrote:
> >>
> >> On Feb 13, 2001, Robert Boehne <address@hidden> wrote:
> >>
> >> > + output=$output_objdir/all_objs${k}.$objext
> >>
> >> This won't work in parallel builds. You must name the temporary
> >> object files after the output.
>
> > By named after, do you mean somthing like "all_objs$output" ?
> > Also, could you explain why this would break a paralell build?
>
> Think of multiple huge libraries being linked concurrently in the same
> directory. All of them would use all_objs0.o, then all_objs1.o, etc.
> If this occurs at the same time, things break.
>
> I suggest $output-${k}.$objext. Oh, and you may add commands to
> remove these temporary files when they've already been used.
> Otherwise, we may end up using quadratic space.
>
> > Ah, yes, I see. I had taken the approach that I should eval more
> > rather than less, but I see that I need to eval _exactly_ the
> > right number of times. I will add a test for this in the final
> > version.
>
--
Robert Boehne Software Engineer
Ricardo Software Chicago Technical Center
TEL: (630)789-0003 x. 238
FAX: (630)789-0127
email: address@hidden
2001-02-15 Robert Boehne <address@hidden>
* ltconfig.in: Add a test to find the approximate limit
to the length of command line arguments. The number
calculated here should always be lower than the actual
limit.
* ltmain.in: Test the length of the command line to be
executed and use an incremtnal linking scheme if the
command is too long to be interpreted without error.
Index: ltconfig.in
===================================================================
RCS file: /cvsroot/libtool/libtool/Attic/ltconfig.in,v
retrieving revision 1.246.2.44
diff -u -r1.246.2.44 ltconfig.in
--- ltconfig.in 2001/02/15 21:55:40 1.246.2.44
+++ ltconfig.in 2001/02/19 22:16:06
@@ -216,6 +216,7 @@
ac_ext=c
libext=a
cache_file=
+max_cmd_len=
## Dependencies to place before and after the object being linked:
predep_objects=
@@ -774,6 +775,21 @@
no_builtin_flag="$ac_cv_prog_cc_no_builtin"
can_build_shared="$ac_cv_prog_cc_can_build_shared"
+# find the maximum length of command line arguments
+echo $ac_n "finding the maximum length of command line arguments... $ac_c" 1>&6
+echo "$progname:@LINENO@:finding the maximum length of command line arguments"
1>&5
+i=0
+testring="ABCDEF"
+while test `$0 --fallback-echo "X$testring" >/dev/null 2>&1` == `echo
"X$testring" >/dev/null 2>&1` &&
+ new_result=`expr "X$testring" : ".*" 2>&1` &&
+ max_cmd_len=$new_result &&
+ test $i != 32 # 1 MB should be enough
+do
+ i=`expr $i + 1`
+ testring=$testring$testring
+done
+echo "$ac_t""$max_cmd_len" 1>&6
+
# Check to see if options -o and -c are simultaneously supported by compiler
echo $ac_n "checking if $compiler supports -c -o file.$objext... $ac_c" 1>&6
$rm -r conftest 2>/dev/null
@@ -2221,6 +2237,9 @@
# Additional compiler flags for building library objects.
pic_flag=$pic_flag
pic_mode=$pic_mode
+
+# What is the maximum length of a command?
+max_cmd_len=$max_cmd_len
# Does compiler simultaneously support -c and -o options?
compiler_c_o=$compiler_c_o
Index: ltmain.in
===================================================================
RCS file: /cvsroot/libtool/libtool/ltmain.in,v
retrieving revision 1.200.2.44
diff -u -r1.200.2.44 ltmain.in
--- ltmain.in 2001/01/24 20:08:34 1.200.2.44
+++ ltmain.in 2001/02/19 22:16:09
@@ -3007,13 +3007,90 @@
else
eval cmds=\"$archive_cmds\"
fi
- IFS="${IFS= }"; save_ifs="$IFS"; IFS='~'
- for cmd in $cmds; do
- IFS="$save_ifs"
- $show "$cmd"
- $run eval "$cmd" || exit $?
- done
- IFS="$save_ifs"
+ expr "X$cmds" : ".*"
+ if test $? -ne 0 || \
+ test `expr "X$cmds" : ".*"` -gt $max_cmd_len ; then
+ # the command line is too long to link in one step, link
incrementally
+ $echo "using incremental linking..."
+ # save the value of $output and $libobjs because we want to use them
later
+ save_libobjs=$libobjs
+ save_output=$output
+ # clear the reloadable object creation command queue and initialize
k to one.
+ concat_cmds=
+ objlist=
+ k=0
+ # loop over the list of objects to be linked
+ for obj in $save_libobjs
+ do
+ expr "X$cmds" : ".*"
+ if test $? -ne 0 || \
+ test `expr "X$test_cmds" : ".*"` -gt $max_cmd_len ; then
+ # the above command is almost too long
+ # increment k, the reloadable object file counter
+ klast=$k
+ k=`expr $k + 1`
+ # set the value of $output to the Kth object file
+ output=$output_objdir/$output-${k}.$objext
+ if test $k -eq 1 ; then
+ # the first file doesn't need to have the previous one linked
in.
+ eval concat_cmds=\"$reload_cmds $objlist\"
+ else
+ # all subsequent reloadable object files will link in the last
one created
+ eval concat_cmds=\"\$concat_cmds~$reload_cmds $objlist
$output_objdir/$output-${klast}.$objext\"
+ fi
+ test_cmds=X
+ objlist=$obj
+ else
+ objlist="$objlist"
+ if test $k -eq 0 ; then
+ # the first file doesn't need to have the previous one linked
in.
+ eval test_cmds=\"$reload_cmds $objlist\"
+ else
+ # all subsequent reloadable object files will link in the last
one created
+ eval test_cmds=\"$reload_cmds $objlist
$output_objdir/$output-${klast}.$objext\"
+ fi
+ fi
+ done
+ # handle the remaining objects by creating one last reloadable
object file
+ klast=$k
+ k=`expr $k + 1`
+ output=$output_objdir/$output-${k}.$objext
+ if test $k != 1 ; then
+ # all subsequent reloadable object files will link in the last one
created
+ eval concat_cmds=\"\$concat_cmds~$reload_cmds $objlist
$output_objdir/$output-${klast}.$objext\"
+ else
+ eval concat_cmds=\"$reload_cmds $save_libobjs\"
+ fi
+ $echo "creating a temporary reloadable object file: $output"
+
+ # loop through the commands generated above and execute them
+ IFS="${IFS= }"; save_ifs="$IFS"; IFS='~'
+ for cmd in $concat_cmds; do
+ IFS="$save_ifs"
+ $show "$cmd"
+ $run eval "$cmd" || exit $?
+ done
+ IFS="$save_ifs"
+
+ libobjs=$output
+ # restore the value of output
+ output=$save_output
+
+ # expand the library linking commands again to reset the value of
$libobjs for incremental linking
+ # Do each of the archive commands.
+ if test -n "$export_symbols" && test -n "$archive_expsym_cmds"; then
+ eval cmds=\"$archive_expsym_cmds\"
+ else
+ eval cmds=\"$archive_cmds\"
+ fi
+ fi
+ IFS="${IFS= }"; save_ifs="$IFS"; IFS='~'
+ for cmd in $cmds; do
+ IFS="$save_ifs"
+ $show "$cmd"
+ $run eval "$cmd" || exit $?
+ done
+ IFS="$save_ifs"
# Restore the uninstalled library and exit
if test "$mode" = relink; then
@@ -3914,6 +3991,32 @@
# done
eval cmds=\"$old_archive_cmds\"
+ `expr "X$cmds" : ".*"`
+ if test $? -ne 0 || \
+ test `expr "X$cmds" : ".*"` -gt $max_cmd_len ; then
+ # the command line is too long to link in one step, link
incrementally
+ $echo "using incremental archive linking..."
+ objlist=
+ concat_cmds=
+ save_oldobjs=$oldobjs
+ for obj in $save_oldobjs
+ do
+ oldobjs="$objlist $obj"
+ objlist="$objlist $obj"
+ eval test_cmds=\"$old_archive_cmds\"
+ `expr "X$test_cmds" : ".*"`
+ if test $? -ne 0 || \
+ test `expr "X$test_cmds" : ".*"` -gt $max_cmd_len ; then
+ # the above command should be used before it gets too long
+ oldobjs=$objlist
+ eval concat_cmds=\"$old_archive_cmds~$concat_cmds\"
+ objlist=
+ fi
+ done
+ oldobjs=$objlist
+ eval concat_cmds=\"$old_archive_cmds~$concat_cmds\"
+ eval cmds=\"$concat_cmds\"
+ fi
fi
IFS="${IFS= }"; save_ifs="$IFS"; IFS='~'
for cmd in $cmds; do
- [MLB] Fix for arg list too long (Finally a patch!), Robert Boehne, 2001/02/13
- Re: [MLB] Fix for arg list too long (Finally a patch!), Alexandre Oliva, 2001/02/14
- Re: [MLB] Fix for arg list too long (Finally a patch!), Robert Boehne, 2001/02/15
- Re: [MLB] Fix for arg list too long (Finally a patch!), Alexandre Oliva, 2001/02/15
- Re: [MLB] Fix for arg list too long (another attempt),
Robert Boehne <=
- Re: [MLB] Fix for arg list too long (another attempt), Alexandre Oliva, 2001/02/20
- Re: [MLB] Fix for arg list too long (another [final?] attempt), Robert Boehne, 2001/02/20
- Re: [MLB] Fix for arg list too long (another [final?] attempt), Alexandre Oliva, 2001/02/22
- Re: [MLB] Fix for arg list too long (another final! attempt), Robert Boehne, 2001/02/22
- Re: [MLB] Fix for arg list too long (another final! attempt), Alexandre Oliva, 2001/02/23