bug#41669: Cross-compiled powerpc64-linux bootstrap-tarballs not reproducible

[Efraim Flashner]

On Fri, Sep 25, 2020 at 11:52:48PM -0700, Chris Marusich wrote:
> Hi everyone,
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > Is this a file we actually need during the bootstrap process? Can we
> > "work around it" by just deleting it?
> 
> That's a good idea.  I tried building %gcc-static with the
> --disable-libstdcxx configure flag (see attached patch), and that caused
> the build of %gcc-static itself to become reproducible on Debian and
> Fedora when using exactly the same inputs.  To clarify, I have recently
> run the following experiments:
> 
> +------+-------------+---------------------+--------------+----------------+---------------+
> | Case |    Build    |      libstdcxx      | substitutes? |     Inputs     |  
>   Result     |
> +------+-------------+---------------------+--------------+----------------+---------------+
> |      |             |                     |              | Built fresh on |  
>   Output     |
> |  1   | %gcc-static |       enabled       |     yes      | Debian, copied |  
>  differs:    |
> |      |             |                     |              |   to Fedora    |  
> libstdc++.a  |
> +------+-------------+---------------------+--------------+----------------+---------------+
> |      |             |                     |              | Re-used inputs |  
> Toy binary   |
> |  2   | toy program |         n/a         |     yes      |   from above   |  
>  does not    |
> |      |             |                     |              |                |  
>   differ     |
> +------+-------------+---------------------+--------------+----------------+---------------+
> |      |             |                     |              | Built fresh on |  
> Output does  |
> |  3   | %gcc-static |      disabled       |     yes      | Debian, copied |  
> not differ   |
> |      |             |                     |              |   to Fedora    |  
>              |
> +------+-------------+---------------------+--------------+----------------+---------------+
> |      |             |                     |              | Built fresh on |  
>   Output     |
> |  4   | %gcc-static |      disabled       |     yes      | Debian, fresh  |  
>  differs:    |
> |      |             |                     |              |   on Fedora    | 
> various files |
> +------+-------------+---------------------+--------------+----------------+---------------+
> |      |             |                     |              | Inputs failed  | 
> Build failed  |
> |  5   | %gcc-static |      disabled       |      no      |  to build on   |  
>   on both    |
> |      |             |                     |              |  both systems  |  
>   systems    |
> +------+-------------+---------------------+--------------+----------------+---------------+
> 
> The "toy program" in case 2 was just this:
> 
>   #include <stdio.h>
>   int main() {
>     printf("Hello");
>     return 0;
>   }
> 
> When I say I "copied" the inputs from Debian to Fedora, I mean just
> that.  I copied stuff like /gnu and /var/guix from Debian to Fedora
> (while guix-daemon was stopped, of course), GC'd just %gcc-static on
> Fedora, and then rebuilt %gcc-static on Fedora so it would use exactly
> the same inputs as were used on Debian.
> 
> The most notable new findings are cases 3 and 4.  Case 5 made me pretty
> sad because I spent almost an entire weekend trying to build Guix from
> source using various binary Guix releases, and in none of the attempts
> was I successful in running "guix pull" or even just "guix environment
> --pure guix" without substitutes, which really surprised me.
> 
> Case 3 confirms Efraim's suggestion: we can fix the libstdc++.a
> reproducibility problem by simply not building libstdc++.a in the first
> place. It also confirms that, when built with --disable-libstdcxx,
> %gcc-static can be built reproducibly on different systems as long as
> exactly the same inputs are used.  Whether or not %gcc-static can be
> used to successfully bootstrap packages on powerpc64-linux when built in
> this way remains to be seen.
> 
> Case 4, unfortunately, demonstrates that there are still other
> reproducibility issues that we have not yet resolved.  Since the only
> difference between cases 3 and 4 is how the inputs were realized, the
> differing %gcc-static output must be caused by the inputs somehow.
> 
> In case 4, the %gcc-static output differed in the following files (the
> hashes on the left are MD5 hashes):
> 
> --8<---------------cut here---------------start------------->8---
> --- /dev/fd/63        2020-09-25 20:35:33.386554595 -0700
> +++ /dev/fd/62        2020-09-25 20:35:33.387554604 -0700
> @@ -1,28 +1,28 @@
> -c9b0dfcbad566c0b8b88df94bb993312  ./bin/c++
> -092823145dc96b9eb81111362f7b4ced  ./bin/cpp
> -c9b0dfcbad566c0b8b88df94bb993312  ./bin/g++
> -e4cc43b7790dcd25f31419bad606b36e  ./bin/gcc
> +8f02302b55643f1c711e472a42fea8bd  ./bin/c++
> +9f1fd993e4f2b796fcc56f0b2f8a47d2  ./bin/cpp
> +8f02302b55643f1c711e472a42fea8bd  ./bin/g++
> +583d1b011a7ba009d7385117dd7a33c8  ./bin/gcc
>  f9d94f4bb61f70d14ea4b2ce73c9be9d  ./bin/gcc-ar
>  01fc2184f99c558771aa8f2fe30b373d  ./bin/gcc-nm
>  da5356ee09ccda4ca06758d056370f7e  ./bin/gcc-ranlib
> -98645f7b00ba185e713915099853fd37  ./bin/gcov
> -37dd62589454703ae7f2eaac1668b66e  ./bin/gcov-dump
> -f3dbc7e0c84a40194af3aa2429444e87  ./bin/gcov-tool
> -c9b0dfcbad566c0b8b88df94bb993312  ./bin/powerpc64-linux-gnu-c++
> -c9b0dfcbad566c0b8b88df94bb993312  ./bin/powerpc64-linux-gnu-g++
> -e4cc43b7790dcd25f31419bad606b36e  ./bin/powerpc64-linux-gnu-gcc
> -e4cc43b7790dcd25f31419bad606b36e  ./bin/powerpc64-linux-gnu-gcc-5.5.0
> +a208bedbfca9c7bd6c27d0d42f7096fe  ./bin/gcov
> +43330e8ae00976b4b3427d2f83b0725e  ./bin/gcov-dump
> +9f37da5e96f147d733eb7195350ae5d2  ./bin/gcov-tool
> +8f02302b55643f1c711e472a42fea8bd  ./bin/powerpc64-linux-gnu-c++
> +8f02302b55643f1c711e472a42fea8bd  ./bin/powerpc64-linux-gnu-g++
> +583d1b011a7ba009d7385117dd7a33c8  ./bin/powerpc64-linux-gnu-gcc
> +583d1b011a7ba009d7385117dd7a33c8  ./bin/powerpc64-linux-gnu-gcc-5.5.0
>  f9d94f4bb61f70d14ea4b2ce73c9be9d  ./bin/powerpc64-linux-gnu-gcc-ar
>  01fc2184f99c558771aa8f2fe30b373d  ./bin/powerpc64-linux-gnu-gcc-nm
>  da5356ee09ccda4ca06758d056370f7e  ./bin/powerpc64-linux-gnu-gcc-ranlib
> -6ed530d13e65c3500b7e7b7cc863afdc  ./libexec/gcc/powerpc64-linux-gnu/5.5.0/cc1
> -24a83af179ca8849da8c64aa854ec8ed  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/cc1plus
> -0c05b45bb926a06c2b09acdb1db9aad0  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/collect2
> +22b72247a5706f090505341263ca1fc2  ./libexec/gcc/powerpc64-linux-gnu/5.5.0/cc1
> +3be618d184038dd30011d6aa8198c0be  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/cc1plus
> +2f31e84c01cc087318d0c7f15b6e3f47  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/collect2
>  417a5b42a26275b2c912db16b9abf73a  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/install-tools/fixincl
>  fd6f80ec9089ddf51f9cac26299e45af  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/install-tools/fixinc.sh
>  a585abbd6a9cdc474564b54fc72e4efa  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/install-tools/mkheaders
>  5071acceb24c0c0e8a423286205ed54c  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/install-tools/mkinstalldirs
> -4e77b773ac45ce8f82a4d21a34063920  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/lto-wrapper
> +5267311e0ed8bb5358e5556dfa205ca6  
> ./libexec/gcc/powerpc64-linux-gnu/5.5.0/lto-wrapper
>  a90ab86f837913280f72beb5310714bc  
> ./lib/gcc/powerpc64-linux-gnu/5.5.0/crtbegin.o
>  0b38aa831c40b6bc4fb22248746d60d8  
> ./lib/gcc/powerpc64-linux-gnu/5.5.0/crtbeginS.o
>  8f62d8795bebd8e87caa3640522ff93b  
> ./lib/gcc/powerpc64-linux-gnu/5.5.0/crtbeginT.o
> --8<---------------cut here---------------end--------------->8---
> 
> These appear to be the same diffs that I reported before - with the
> notable exception that libstdc++.a now is missing from the list of
> differing files (hooray!).
> 
> Going forward, I'm not sure how best to investigate the inputs to find
> out what's causing the differences.  I just know I really, really,
> really don't want to rebuild everything multiple times, since it takes
> hours/days.  If you have any creative ideas for how to speed up the
> investigation, I'm all ears.
> 

Does diffoscope provide any useful clues as to what's going on? I don't
remember if gcc-5.5.0 was reproducible when I created the aarch64
bootstrap binaries.

> -- 
> Chris

> From e3d1778a86dfd171d59d91eb01417faaf63dfa17 Mon Sep 17 00:00:00 2001
> From: Chris Marusich <cmmarusich@gmail.com>
> Date: Sat, 19 Sep 2020 14:25:43 -0700
> Subject: [PATCH] gnu: Disable libstdc++ in bootstrap GCC.
> 
> Fixes part of: <https://bugs.gnu.org/41669>.
> 
> * gnu/packages/make-bootstrap.scm (%gcc-static) [#:configure-flags]: Add
> --disable-libstdcxx to disable building the libstdc++-v3 directory.
> ---
>  gnu/packages/make-bootstrap.scm | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/gnu/packages/make-bootstrap.scm b/gnu/packages/make-bootstrap.scm
> index b2d3e2a326..8632d63c21 100644
> --- a/gnu/packages/make-bootstrap.scm
> +++ b/gnu/packages/make-bootstrap.scm
> @@ -487,6 +487,10 @@ for `sh' in $PATH, and without nscd, and with static NSS 
> modules."
>                     ;; Make sure gcc-nm doesn't require liblto_plugin.so.
>                     "--disable-lto"
>  
> +                   ;; In this GCC version, libstdc++.a is not reproducible:
> +                   ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669
> +                   "--disable-libstdcxx"
> +
>                     "--disable-shared"
>                     "--disable-plugin"
>                     "--disable-libmudflap"
> -- 
> 2.26.2
> 




-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

signature.asc
Description: PGP signature