Re: [PATCH v2] build: Option for building all tools in a single binary

[Pádraig Brady]

On 07/02/2014 04:34 PM, Pádraig Brady wrote:
> On 07/02/2014 01:44 AM, Alex Deymo wrote:
>> Hi again,
>>
>> Sorry for the delay with this patch. Here is patch v7 attached.
>>
>> *** symlinks vs shebangs:
>>
>> * Other projects use symlinks to have a multicall binary; including busybox 
>> and toolbox which cover similar functionality as coreutils; but not limited 
>> to them: lvm (vgcreate, pvcreate, etc), xz (xzcat, unxz point to "xz") and 
>> others I'm not aware of.
>>
>> * People using symlinks do not protect against the symlink to symlink case; 
>> they just rely on argv[0]. symlink to symlink is not the only way to provide 
>> a different argv[0]; you can pass whatever you want as the first argument of 
>> argv when you call exec().
>>
>> * The scheme to follow symlinks to determine which one was the last symlink 
>> requires to accurately reproduce the path search algorithm used by exec() on 
>> the platform we are running; and requires the make a couple of syscalls just 
>> to know what's the tool to call, yet still doesn't solve the whole case.
>>
>> * Shebangs allow you to have a symlink to them and it still works without 
>> extra work. On some systems, we can change the value of /proc/$PID/cmdline 
>> and other files to not break use cases like "killall foo" or the result of 
>> "ps".
>>
>>
>> In general, if you want to use --enable-single-binary is because size is a 
>> concern; and in those cases you know your environment (you are deploying an 
>> embedded system with a particular kernel). You might not care about 4MB in a 
>> multi GB linux distro and there's people out there still doing symlink to 
>> symlink that eventually will get their scripts right; but not yet.
>>
>> So, I added an option where you can choose at configure time if you want to 
>> install symlinks or shebangs (--enable-single-binary=shebangs for example). 
>> I tested this patch on the ChromiumOS builders and it worked (we managed to 
>> build a builder environment and images for amd64, x86 and arm).
>>
>> In the shebang case, the program will attempt to modify /proc/$PID/cmdline 
>> by using prctl() if available. There are other ways to modify argv[] on 
>> other platforms (look at the sendmail or postgresql case for example), so 
>> eventually we should migrate that functionality to a gnulib file and make it 
>> work on all the supported platforms... but for now, you still have the 
>> choice to use symlinks if that works for you.
>>
>> Let me know if you have other questions about this patch and happy canada 
>> day for the canadians out there on this list. 
>>
>> deymo.
>>
> 
> Excellent work.
> 
> One small issue I noticed (in symlinks mode at least)
> was that `make syntax-check` causes all the
> tools to be remade as stand-alone binaries.
> This is related to `make src/$binary` replacing
> the symlink with a stand-alone binary.
> 
> Note that isn't a supported end user mode anyway,
> so it's not blocking, though nice to avoid.
> 
> As an aside, the reason end users can not do:
>   make src/$binary
> is due to our use of BUILT_SOURCES of which the automake manual says:
>   "you cannot use BUILT_SOURCES if the ability to run
>    ‘make foo’ on a clean tree is important to you."
> coreutils uses BUILT_SOURCES, as the mandatory configure
> step is expensive compared to the build itself,
> so there is not much advantage to supporting that.
> 
> A more problematic issue is that program prefix isn't supported.
> i.e.: ./configure --enable-single-binary --program-prefix=g
> 
> BTW I've attached some spelling fixes which I've rolled in here.
> 
> thanks!
> Pádraig.
> 

Getting ready to merge this.
Attached are the following adjustments:

commit 6a7418ca215f8d99107948b95ff5d3d7672d7a2d
Author: Pádraig Brady <address@hidden>
Date:   Fri Jul 4 15:59:17 2014 +0100

    build: use the installed program name in --help and /proc/$PID/comm

    Note we still use the internal coreutils names for --coreutils-prog,
    independent of what's used on the system due to --prefix-prog etc.
    So adjust the argv appropriately so that --help and /proc/$PID/comm
    use the correct command name for the current install.

commit 1a7c3b46056f009624a2d1e18a7bbe70e5d67eff
Author: Pádraig Brady <address@hidden>
Date:   Fri Jul 4 15:19:54 2014 +0100

    tests: avoid false failure without a working strip

    * tests/install/basic-1.sh: Really avoid using ginstall strip
    functionality if there is an issue with the independent strip command.
    This avoids a false failure with --enable-single-binary=shebangs.

commit c53e8a03df916c7757b371f95435380087afecd0
Author: Pádraig Brady <address@hidden>
Date:   Fri Jul 4 15:09:17 2014 +0100

    build: support directly calling coreutils with shebangs enabled

    When called with a shebang, the path of the shebang script
    will be added to the command line.  Therefore there is a different
    number of args to strip depending on whether we're calling through
    the shebang script or directly calling `coreutils --coreutils-prog`.
    Also adjust argv[0] to just the identified program name.
    I.E. without the path.  This is usually correct, and allows
    error string matching in the tests to work as expected.

commit 1ec17595c0c80757a794d9ffa68ed9f64c6a195f
Author: Pádraig Brady <address@hidden>
Date:   Fri Jul 4 15:04:49 2014 +0100

    build: support --program-prefix et. al. with --enable-single-binary

    * Makefile.am: Honor any user configured $transform.
    This also uses 'install' rather than 'ginstall' as
    the destinary path.

multicall-adjustments.patch
Description: Text Data