bug#49439: grafts cause “guix environment” to get killed with OOM

[Ludovic Courtès]

Hi,

Sarah Morgensen <iskarian@mgsn.dev> skribis:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> With a recent version of Guix, “guix environment” will not 
>> terminate on its own, keeps the CPU busy, and gets killed when the 
>> system eventually runs out of memory.
>>
>> $ guix describe -f channels
>>
>> (list (channel
>>         (name 'guix)
>>         (url "/home/rekado/dev/gx/branches/master")
>>         (commit
>>           "685cfdec94e5e48c4ad28de53466a28dfc258edb")))
>>
>>
>> $ guix environment pigx-scrnaseq
>> [wait until it gets killed]
>
> I can reproduce this with pigx-scrnaseq as well a number of other
> packages (listed below).
>
> $ ./pre-inst-env guix describe -f channels
> (list (channel
>         (name 'guix)
>         (url "/home/sarah/guix")
>         (commit
>           "3217a04b0352c2dd13323257b369604eeabfccc3")))
>
> Does not complete within 5 minutes:

What hardware are you using?

Here’s what I observe (with everything already in store and on a hot
cache, with an i7 CPU):

--8<---------------cut here---------------start------------->8---
$ guix describe
Generacio 188   Jul 25 2021 12:47:29    (nuna)
  guix a92dfbc
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: a92dfbce30777de6ca05031e275410cf9f56c84c
$ time GUIX_PROFILING=gc guix environment pigx-scrnaseq --search-paths 
--no-grafts >/dev/null
Garbage collection statistics:
  heap size:        160.31 MiB
  allocated:        1440.70 MiB
  GC times:         39
  time spent in GC: 4.51 seconds (46% of user time)

real    0m7.534s
user    0m9.747s
sys     0m0.167s
$ time GUIX_PROFILING=gc guix environment pigx-scrnaseq --search-paths  
>/dev/null
Garbage collection statistics:
  heap size:        168.31 MiB
  allocated:        2111.71 MiB
  GC times:         53
  time spent in GC: 6.92 seconds (45% of user time)

real    0m12.100s
user    0m15.517s
sys     0m0.198s
--8<---------------cut here---------------end--------------->8---

Commit 78daf9e02e5bc51f91488d8237cab2050cc060cf optimizes
‘coalesce-duplicate-inputs’, which was quadratic in the number of inputs (!).
After that change, I get:

--8<---------------cut here---------------start------------->8---
$ time GUIX_PROFILING=gc ./pre-inst-env guix environment pigx-scrnaseq 
--search-paths --no-grafts  >/dev/null
Garbage collection statistics:
  heap size:        168.31 MiB
  allocated:        716.58 MiB
  GC times:         24
  time spent in GC: 2.65 seconds (40% of user time)

real    0m5.720s
user    0m6.708s
sys     0m0.149s
$ time GUIX_PROFILING=gc ./pre-inst-env guix environment pigx-scrnaseq 
--search-paths  >/dev/null
Garbage collection statistics:
  heap size:        160.31 MiB
  allocated:        1387.96 MiB
  GC times:         42
  time spent in GC: 5.87 seconds (43% of user time)

real    0m10.821s
user    0m13.513s
sys     0m0.198s
--8<---------------cut here---------------end--------------->8---

Could you tell me if it improves the situation for you?

It’s not the end of the road, but it should be an improvement.

Thanks,
Ludo’.