Re: speed up large library linking

[Ralf Wildenhues]

Hi Peter,

* Peter O'Gorman wrote on Fri, May 20, 2005 at 03:29:36PM CEST:
> Ralf Wildenhues wrote:
> >
> >How may I convince you?
> 
> I am not sure that you can convince me to like the temporary file usage, 
> and the use of utilitles like paste and join which are neither mentioned in 
> the GNU coding standards document or autocnf's portable shell section. The 
> fact that the files stick around if the user interrupts the process only 
> makes mattters worse. These patches are very destabilizing and, in my 
> opinion, make libtool harder to maintain.

I'll try to give a couple of technical comments only.
You mention several issues:

- Use of temp files.
Note that we've used them already, for years, while executing
$relink_command in func_mode_link.

- Chance of leftover temp files.
The way these are removed may be changed to something more appropriate.
We should apply that to the $relink_command case as well then.

- Use of `non-standard' utilities.
I have thought about the possibility of replacing the whole `fold' and
`split'-using stuff with `xargs' (which _is_ mentioned in GCS), but am
not yet sure whether that works.  Should I look into it?

`join' and `paste' are used in the rename algo only.  As already
mentioned, we could add the old algorithm as a fallback.

All of the utilities are required by POSIX/SUSv3.

> Now I have thought about this amost every day since you first posted this 
> patch and the speedup is very convincing, I am sure that for you it easily 
> convinces you that the patch is necessary. For me it kind of balances the 
> scale, at one point last week I was going to give approval to commit when I 
> realized that libjava is atypical libtool usage.

Yes, this was my first thought as well.  In fact, I even mentioned that
in [1].  At the time I believed, however, that there was a reason for
libjava to consist of so many objects.

> It's massive number of objects exceed kernel limits for command line
> length on many platforms, I have never seen another library do that.

I know of one other package which comes close on some systems, and which
also benefits of these changes.

> I will note however that Paolo recently committed a patch to gcc to
> compile libjava a directory at a time rather than a file at a time,
> reducing the compile time for libjava dramatically.

I have seen it and tried it as well.  It makes my link mode patches
completely unnecessary for libjava.

> Some portions of this patch eliminate unnecessary loops, or have other 
> optimizations, I like them, just can't deal with the tempfiles.

Another note: I could not figure out a way to write OUTPUT_LONG_LIST
without a temp file, because if we use a pipe for the first $ECHO, we
cannot find out portably whether it failed(?).

> Sorry, hope my explination makes some sense. I encourage other maintainers 
> to make their feelings clear.

No need to feel sorry at all.  :)

Cheers,
Ralf

[1] http://lists.gnu.org/archive/html/bug-libtool/2005-05/msg00272.html