[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#49439: grafts cause “guix environment” to get killed with OOM

From: Ludovic Courtès
Subject: bug#49439: grafts cause “guix environment” to get killed with OOM
Date: Thu, 29 Jul 2021 00:03:46 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)


Ludovic Courtès <ludo@gnu.org> skribis:

> Thinking about it, the grafts code depends on what’s in the store: when
> nothing is in the store, it bounces to the “build handler”, which
> accumulates the list of missing store items, until it starts building
> them.

So I can reproduce the problem Ricardo and you initially reported by

  ./pre-inst-env guix environment pigx-scrnaseq --search-paths

after removing some of the ungrafted store items with:

  guix gc -D $(guix build r-rlang --no-grafts) \
    $(guix gc --references $(guix build pigx-scrnaseq --no-grafts))

The seemingly endless CPU usage and unbound memory use comes from the
‘build-accumulator’ build handler.  Here, the graph of ‘pigx-scrnaseq’
has many nodes, and many of them depend on, say, ‘r-rlang’.  Thus, when
‘r-rlang’ is not in the store, the grafting code keeps asking for it by
calling ‘build-derivations’, which aborts to the build handler; the
build handler saves the .drv file name and the continuation and keeps
going.  But since the next package also depends on ‘r-langr’, we abort
again to the build handler, and so on.

The end result is a very long list of <unresolved> nodes, probably of
this order in this case:

  $ guix graph -t reverse-package r-rlang |grep 'label = "'|wc -l

Presumably, the captured continuations occupy quite a bit of memory,
hence the quick growth.

I suppose one solution is to fire suspended builds when the build
handler sees a build request for a given derivation for the second time.
It needs more thought and testing…


PS: Did you know ‘pigx-scrnaseq’ has twice as many nodes as

      $ guix graph -t bag pigx-scrnaseq |grep 'label = "'|wc -l
      $ guix graph -t bag libreoffice |grep 'label = "'|wc -l

    That makes it a great example to study and fix scalability issues!

reply via email to

[Prev in Thread] Current Thread [Next in Thread]