Error "Arguments list too long" in gold test many_sections_check

[Ilya Palachev]

Good day, dear binutils community.

During the build of the latest version of binutils of version 2.24.51 we 
have obtained the following build error:

[  944s] gcc -DHAVE_CONFIG_H -I. -I../../gprof  -DDEBUG -I../bfd -
I../../gprof/../include -I../../gprof/../bfd  -I. -
DLOCALEDIR="\"/usr/share/locale\""  -W -Wall -Wstrict-prototypes -Wmissing-
prototypes -Wshadow -Werror -O2 -g2 -feliminate-unused-debug-types -pipe -
Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-
buffer-size=32 -Wformat -Wformat-security -fmessage-length=0 -march=armv7-a
-mtune=cortex-a8 -mlittle-endian -mfpu=neon -mfloat-abi=softfp -D__SOFTFP__
-g -Wno-error -MT symtab.o -MD -MP -MF .deps/symtab.Tpo -c -o symtab.o ../..
/gprof/symtab.c
[  944s] make[4]: Entering directory `/home/abuild/rpmbuild/BUILD/binutils-
2.24.51/build-dir/gold/testsuite'
[  944s] (for i in `seq 1 70000`; do \
[  944s]    echo "int var_$i __attribute__((section(\"section_$i\"))) =
$i;"; \
[  944s]  done) > many_sections_define.h.tmp
[  944s] make[4]: execvp: /bin/sh: Argument list too long
[  944s] make[4]: *** [many_sections_define.h] Error 127
[  944s] make[4]: Leaving directory `/home/abuild/rpmbuild/BUILD/binutils-2.
24.51/build-dir/gold/testsuite'
[  945s] make[4]: Entering directory `/home/abuild/rpmbuild/BUILD/binutils-
2.24.51/build-dir/gold'

The full build log and configuration can be obtained from the following 
package in Tizen's Open Build Service:
https://build.tizen.org/package/show?package=binutils_fail&project=devel%3Aarm_toolchain%3AMobile%3Atest

1. We have found that the error happens during the execution of Makefile 
for gold/testuite, more precisely - during
the making of the following Makefile's target:

many_sections_define.h:
       (for i in `seq 1 70000`; do \
          echo "int var_$$i __attribute__((section(\"section_$$i\"))) = 
$$i;"; \
        done) > address@hidden
       mv -f address@hidden $@

It's interesting that the problem does not deal with long argument lists 
at all.
At first I have tried to replace `seq 1 70000` with bash loops, but soon 
it was realized that even if the rule is totally omitted, the error 
still occurs.
Even with the following change:

many_sections_define.h:
    true

we have the following build error:

/usr/bin/true: Argument list too long

It's obvious that the problem is connected with our build environment, 
especially because it is obtained only during
the build for the architectures "armv7l" and "aarh64", but for 
architectures "i686" and "x86_64" the build is succeeded.

Maybe it's connected with wrong build configuration of bash, or even 
it's a qemu bug, since all the build is run under emulator.

2. We thought that the problem appears, because "make" calls "bash" with 
too long command line arguments list.

Here is the end output of "make -j1 -d" (for modified Makefile, where 
all commands of rule were replaced with one command "true")::

Considering target file `all'.
 File `all' does not exist.
  Considering target file `many_sections_define.h'.
   File `many_sections_define.h' does not exist.
   Finished prerequisites of target file `many_sections_define.h'.
  Must remake target `many_sections_define.h'.
Invoking recipe from Makefile:4978 to update target 
`many_sections_define.h'.
ls
Putting child 0x9d59d68 (many_sections_define.h) PID 19253 on the chain.
Live child 0x9d59d68 (many_sections_define.h) PID 19253
/usr/bin/true: Argument list too long
Reaping losing child 0x9d59d68 PID 19253
make: *** [many_sections_define.h] Error 255
Removing child 0x9d59d68 PID 19253 from chain.

I have replaced the binary "/bin/true" with the following perl script:

#!/usr/bin/perl

print "$#ARGV\n";
print "@ARGV";

And the following output was obtained (for modified Makefile, where all 
commands of rule were replaced with one command "true"):

bash-4.2# set -x
bash-4.2# make -j1
+ make -j1
true
-1

true

So "make" calls binary with 0 number of arguments. Is this conclusion true?

3. After that we tried to disable test "many_sections_check" during 
which the error occurs. The following lines in Makefile.am were commented:

#check_PROGRAMS += many_sections_test
#many_sections_test_SOURCES = many_sections_test.cc
#many_sections_test_DEPENDENCIES = gcctestdir/ld
#many_sections_test_LDFLAGS = -Bgcctestdir/ -rdynamic
#many_sections_test_LDADD =

#BUILT_SOURCES += many_sections_define.h
#MOSTLYCLEANFILES += many_sections_define.h
#many_sections_define.h:
#       (for i in `seq 1 70000`; do \
#          echo "int var_$$i __attribute__((section(\"section_$$i\"))) = 
$$i;"; \
#        done) > address@hidden
#       mv -f address@hidden $@

#BUILT_SOURCES += many_sections_check.h
#MOSTLYCLEANFILES += many_sections_check.h
#many_sections_check.h:
#       (for i in `seq 1 1000 70000`; do \
#          echo "assert(var_$$i == $$i);"; \
#        done) > address@hidden
#       mv -f address@hidden $@

#check_PROGRAMS += many_sections_r_test
#many_sections_r_test.o: many_sections_test.o gcctestdir/ld
#       gcctestdir/ld -r -o $@ many_sections_test.o
#many_sections_r_test: many_sections_r_test.o gcctestdir/ld
#       $(CXXLINK) -Bgcctestdir/ many_sections_r_test.o $(LIBS)

Our default spec-file is written so that autotools are not called at 
all, and we call "configure" script distributed inside official binutils 
tarball.

This change in Makefile.am cannot disable the build of test 
"many_sections_check", until we run autoreconf inside the directory "gold".

We run the autoreconf inside directory "gold" and the build began to 
fail with the following errors:

[  108s] checking command to parse nm output from gcc object... yes
[  108s] configure: error: "unsupported target alpha-tizen-linux-gnu"

and also for other non-standard architectures.

How can we re-generarate file gold/testsuite/Makefile.in without a harm 
to other architectires?


----------

So, we have 3 questions:

1. How to debug such problems, can you advise something?

2. Does the problem occur because "make" calls "bash" with too long list 
of arguments? Or the output of perl script listed above shows opposite 
conclusion?

3. How to disable one test in gold/testsuite/Makefile.am without harm to 
other architectures? Is it possible to re-generate Makefile.in's in 
current binutils release?
What's the proper way of binutils build? Only "configure" is allowed? 
Are binutils designed so that these files can be changed only by 
developers of binutils,
and they make manual changes in auto-generated files?


---------

Best regards,

Ilya Palachev