[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: out-of-control GC
From: |
Hans Åberg |
Subject: |
Re: out-of-control GC |
Date: |
Mon, 11 Sep 2017 20:13:25 +0200 |
> On 9 Sep 2017, at 21:47, Linas Vepstas <address@hidden> wrote:
>
> I've been experiencing problems with guile GC for a while now, that I've
> mostly been able to ignore, but that are becoming debilitating, and I need
> help, or need to find some solution.
>
> Background: I have a C++ library that has guile bindings; it is wrapped up
> as a guile module. It manages a lot of data, anywhere from a few gigabytes,
> to a over a hundred gigabytes. Almost all this RAM usage should be
> invisible, opaque to guile: so, for example, while my library is managing
> 5 or 10 GBytes, guile itself is managing maybe 20 or 50 or 100MBytes,
> according to (gc-stats). This is normal and expected. Some of my programs
> have guile use considerably more than this, in the multi-GByte range
>
> However, the larger the RAM usage, the more frequently guile GC seems to
> run, and the longer it takes, so that by the time my dataset is 100GBytes,
> guile is using maybe 1GBytes, and is spending 99% of its time in GC. My app
> slows to a crawl, and nothing gets done. I need to get GC under control,
> because it is running pointlessly, because its clear that (almost) nothing
> can be reclaimed, (usually) nothing is being reclaimed.
Does it have to do with this issue [1], that the Boehm GC initially uses a
small amount of heap. One can use say
export GC_INITIAL_HEAP_SIZE=50G
on the shell command line before running the program.
1. https://github.com/Macaulay2/M2/issues/500
Re: out-of-control GC,
Hans Åberg <=
Re: out-of-control GC, Kjetil Matheussen, 2017/09/11