[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Compiler memory consumption
From: |
Ludovic Courtès |
Subject: |
Re: Compiler memory consumption |
Date: |
Tue, 16 May 2017 22:45:58 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) |
Heya,
Andy Wingo <address@hidden> skribis:
>> Also, for reference, loading python.go peaks at 315M RSS:
>>
>> $ \time ./pre-inst-env guile -c '(use-modules (gnu packages python))'
>> 0.18user 0.02system 0:00.18elapsed 112%CPU (0avgtext+0avgdata
>> 315648maxresident)k
>> 0inputs+0outputs (0major+7784minor)pagefaults 0swaps
>
> But this I don't understand. If I do a ./pre-inst-env guile and then
> load (gnu packages python), I get a 20MB heap size, a 35MB total private
> dirty memory and 52MB clean shared memory. (Measured using smaps via
> https://wingolog.org/pub/mem_usage.py). This is at commit
> 60c9e80444421c412ae3d0e7b4b224ef0e32947f.
>
> I just built the "time" package and I see similar numbers here. I can
> only think that the "time" package's numbers are bogus.
As discussed on IRC, I just checked and ‘time’ uses the ‘wait3’ syscall
to get the process’s resource usage info once it has completed. In
libc.info it’s described like this:
‘long int ru_maxrss’
The maximum resident set size used, in kilobytes. That is,
the maximum number of kilobytes of physical memory that
PROCESSES used simultaneously.
>> When compiling python.scm #:to 'cps, we end up with 1G max RSS in 6s.
>
> Measured with time? If this is the case it could be that python.scm is
> just a lot of code. Any compiler would take a lot if the IR size is 1
> GB.
>
>> The only conclusion I can draw is that cps-to-bytecode compilation seems
>> to be responsible for most of the memory consumption.
>
> This is possible but I am not there yet. I don't see why compiling this
> file to CPS should cause memory usage of 1GB. That is 9000 memory words
> per textual line -- simply too much.
>
> Many unknowns here!
Yes. :-/
Ludo’.