bug#48963: "guix build guix" halts after exhausting memory

[Ludovic Courtès]

Hi,

Julien Lepiller <julien@lepiller.eu> skribis:

> I tried updating my system on my armhf board (2GB of RAM), but during
> "guix system reconfigure", guix tries to build itself (the guix package
> from (gnu packages package-management)). This package uses too much
> memory to build, and I start getting GC warnings like so:
>
> GC Warning: Out of memory - trying to allocate requested amount (552
> bytes)...
> GC Warning: Header allocation failed: dropping block
> GC Warning: Out of Memory! Heap size: 2571 MiB. Returning NULL!
> Warning: Unwind-only out of memory exception; skipping pre-unwind
> handler.
>
> after some more progress and a lot more warnings, the build stops (CPU
> is at 2%), but the memory is not freed. The build stayed stuck for a
> long time until I decided to cancel it.

This is ridiculous.  :-/

> Can we instead break the build (at the Makefile level) into multiple
> smaller chunks, that require less memory, in the same way (guix self)
> works?

Yes, that’s a good idea.

Could you check the extent to which the attached patch helps on this
machine?

It doesn’t split as much as (guix self) does, only in three pieces, but
hopefully that helps a bit.  A side effect is that progress report is
now off, but we can fix it later.

Thanks,
Ludo’.

diff --git a/Makefile.am b/Makefile.am
index aa21b5383b..758d8b9b8a 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -663,7 +663,11 @@ CLEANFILES =                                       \
 # the whole thing.  Likewise, set 'XDG_CACHE_HOME' to avoid loading possibly
 # stale files from ~/.cache/guile/ccache.
 %.go: make-go ; @:
-make-go: $(MODULES) guix/config.scm $(dist_noinst_DATA)
+make-go: make-core-go make-packages-go make-system-go
+
+define guile-compilation-rule 
+
+$(1): $(2)
        $(AM_V_at)echo "Compiling Scheme modules..." ;                  \
        unset GUILE_LOAD_COMPILED_PATH ;                                \
        XDG_CACHE_HOME=/nowhere                                         \
@@ -671,7 +675,19 @@ make-go: $(MODULES) guix/config.scm $(dist_noinst_DATA)
        $(top_builddir)/pre-inst-env                                    \
        $(GUILE) -L "$(top_builddir)" -L "$(top_srcdir)"                \
          --no-auto-compile                                             \
-         -s "$(top_srcdir)"/build-aux/compile-all.scm $^
+         -s "$(top_srcdir)"/build-aux/compile-all.scm $$(filter %.scm,$$^)
+
+.PHONY: $(1)
+
+endef
+
+MODULES_CORE     = $(filter guix/%,$(MODULES))
+MODULES_PACKAGES = $(filter gnu/packages/%,$(MODULES))
+MODULES_SYSTEM   = $(filter-out gnu/packages/%,$(filter gnu/%,$(MODULES)))
+
+$(eval $(call guile-compilation-rule,make-core-go,$(MODULES_CORE) 
guix/config.scm $(dist_noinst_DATA)))
+$(eval $(call guile-compilation-rule,make-packages-go,$(MODULES_PACKAGES) 
make-core-go))
+$(eval $(call guile-compilation-rule,make-system-go,$(MODULES_SYSTEM) 
make-packages-go make-core-go))
 
 SUFFIXES = .go