Re: Evading the command line length limit (on Linux)

[Paul Smith]

On Wed, 2009-07-29 at 22:51 +0200, Ralf Wildenhues wrote:
> 1) One possibility would be to drop -c and pipe the command line to sh
> on standard input.  Just like parallel make, this would prevent using
> the standard input of the make process within the child command process.
> 
> 2) Another possibility would be a hack like the following: split the
> long line into N substrings, each below the argument limit, and invoke
> 
>   sh -c 'eval "${1}${2}...${10}...${N}"' sh quoted-substring1 ...
> 
> This changes the following user-observable semantics:
> 
> - the positional parameters have different settings; this can be
>   fixed with a 'set x; shift;' at the start of the eval'ed string,
> - LINENO is not reliable in eval with some shells.
> - ${10} and up are not portable to pre-Posix shells like Solaris sh and
>   the heirloom shell under Linux
>   (See autoconf.info 'Limitations of Builtins' for more details on these
>   issues.)
> 
> I don't think these cause big problems in practice, more so if we go
> this way only with command line lengths that would otherwise have failed
> anyway, and only on Linux systems.

Hrm.  I have to say I don't think I like option #2.  It has "special
hack" written all over it.

Not only will it only work on Linux but the ${10} thing ties it to bash,
which means even for Linux-based for systems like Ubuntu where /bin/sh !
= bash it's not so great.

Remember that we actually have this problem in spades on other systems,
like Windows, where the command line length if far SHORTER than Linux.
I'd prefer to find some solution which solves the problem for everyone,
rather than adding some pretty obscure special-case situations.

I'd prefer to use something closer to #1, which is more portable (I
think).  However, it does break POSIX and could break some makefiles
that rely on stdin being available.  Maybe the thing to do is require
the user to specify which rules would be treated in this special way.
That makes it less automatic, which is too bad, but the length of the
command line is pretty static so the person writing the makefile should
know when it's needed.

Alternatively if we can find a way to do it automatically in a portable
way that would be OK too, since the alternative would be to fail anyway.
For Windows and other systems we can probably just choose a hardcoded
number; for UNIX systems it's more complicated.

There have been various ideas floating around about allowing make to
create/use @-files, which a lot of utilities are supporting now.  Maybe
that is a solution here as well?

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <address@hidden>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.mad-scientist.us
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist