bug#41669: Cross-compiled powerpc64-linux bootstrap-tarballs not reproducible

[Chris Marusich]

Hi,

I tried to do some experiments to see if this problem happens with the
current GCC (version 10).

I built GCC 10 (not cross-compiling) on an x86_64 system using Guix with
substitutes on Debian.  (I tried without substitutes, too, but some of
the dependencies failed to be built for unrelated reasons.)  I then
manually copied the /gnu/store and related files (except for the GCC 10
output paths) from Debian onto a Fedora machine, and I rebuilt GCC 10
there using Guix (again, not cross-compiling).  The output on Fedora was
identical to that of Debian.  Of course, the configuration Guix uses to
build GCC 10 is a bit different from the one used to (cross-)build the
powerpc64-linux bootstrap GCC, but it's still an interesting data point.
In particular, GCC 10's libstdc++.a was identical on Debian and Fedora,
so I suppose maybe they've fixed that issue in the more recent versions.

I also tried to use Guix (the current version, from master branch - I
ran guix pull today) to cross-build gcc-10 for the powerpc64-linux-gnu
target on both Debian and Fedora x86_64 systems, starting from scratch
with substitutes enabled:

guix build --target=powerpc64-linux-gnu -e '(@ (gnu packages gcc) gcc-10)'

On both Debian and Fedora, the build of gcc-10.2.0.drv failed with the
following error:

checking for -fPIC -shared... yes
configure: error: 
   Building GCC with plugin support requires a host that supports
   -fPIC, -shared, -ldl and -rdynamic.

This basically just means that we can't cross-build gcc-10 for
powerpc64-linux-gnu out of the box on x86_64 with current Guix.  I was
hoping that the builds would succeed, and I would be able to find out if
cross-building gcc-10 in this way would create non-reproducible
artifacts.  I was hoping maybe I could ask for help from the GCC
community if that were the case.  But since it doesn't even build, the
results of that experiment were not very useful.

It's been almost half a year now, and we're not really any closer to
figuring out why the cross-built GCC bootstrap binary is
non-reproducible.  It seems counter-productive to obsess about making
this specific binary reproducible, although I wish it could be so.

What do you think about using the bootstrap binaries we built half a
year ago, and proceed with bootstrapping efforts?  To be totally honest,
I'm feeling pretty exhausted by this bug, since I have spent so many
days trying to unravel it, and I haven't made any significant progress.
With no clear end in sight, I would really prefer to move on instead of
blocking the entire bootstrapping effort on this reproducibility bug.
The reproducibility of the bootstrap binaries is important, but simply
having any bootstrap binaries at all is also important.  I think I have
done my due diligence to try making them reproducible.  Most of them
are, but I just can't figure out why GCC isn't.  I think it would be
best to proceed with the binaries we have.

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Chris,
>
> Chris Marusich <cmmarusich@gmail.com> skribis:
>
>> From e3d1778a86dfd171d59d91eb01417faaf63dfa17 Mon Sep 17 00:00:00 2001
>> From: Chris Marusich <cmmarusich@gmail.com>
>> Date: Sat, 19 Sep 2020 14:25:43 -0700
>> Subject: [PATCH] gnu: Disable libstdc++ in bootstrap GCC.
>>
>> Fixes part of: <https://bugs.gnu.org/41669>.
>>
>> * gnu/packages/make-bootstrap.scm (%gcc-static) [#:configure-flags]: Add
>> --disable-libstdcxx to disable building the libstdc++-v3 directory.
>
> [...]
>
>> +                   ;; In this GCC version, libstdc++.a is not reproducible:
>> +                   ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669
>> +                   "--disable-libstdcxx"
>
> Does it have any effect with GCC > 4.7?  My understanding is that it
> builds its libstdc++ no matter what.
>
> Also, if it’s just libstdc++.a that’s problematic (ordering issue in the
> .a archive?), perhaps we can use --disable-shared?
>
> My 2¢ (I didn’t follow the whole discussion),
> Ludo’.

Actually, --disable-shared is already present in the configure options.
My understanding is that libstdc++.a is a statically linked library
(perhaps I am mistaken...?), so I don't see why the presence or absence
of --disable-shared would affect it.  I thought that option was just
supposed to control whether or not to build shared libraries.

Efraim Flashner <efraim@flashner.co.il> writes:

> On Fri, Sep 25, 2020 at 11:52:48PM -0700, Chris Marusich wrote:
>> Hi everyone,
>> 
>> Efraim Flashner <efraim@flashner.co.il> writes:
>> 
>> > Is this a file we actually need during the bootstrap process? Can we
>> > "work around it" by just deleting it?

I've spent all of my spare Guix time trying to debug this
reproducibility issue first, and half a year has passed without progress
as a result.  I think we should use the bootstrap binaries we built half
a year ago, and move on with life.

At this point, it might even make more sense to try bootstrapping for
powerpc64le instead of powerpc64, since the rest of the world seems to
be gravitating toward the little-endian variant on POWER9 hardware, and
thus various programs out there are more likely to be better tested on
powerpc64le than powerpc64.

In any case, I don't think we should wait any longer.

-- 
Chris

signature.asc
Description: PGP signature