Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

Hi,

address@hidden (Ludovic Courtès) writes:

>   * the heap gets more and more underutilized, although it seems to
>     eventually stabilize somewhere around 5% (!) utilization (i.e.,
>     total/alive = 20);

The main reason appears to be that the size of individual segments grows
exponentially (it's computed in `scm_i_get_new_heap_segment ()').  This
can be seen by regularly dumping the result of the following expression:

  (map (lambda (elem)
         (abs (- (cdr elem) (car elem))))
       (assoc-ref (gc-stats) 'cell-heap-segments))

The default value of `scm_max_segment_size' is
`SCM_DEFAULT_MAX_SEGMENT_SIZE' and is as large as 20MiB (!).  Setting it
to 64KiB, using the `GUILE_MAX_SEGMENT_SIZE' environment variable,
reduces the total heap size (not shown here) by a factor of 2, and
significantly improves the total-to-alive heap ratio:

A side effect of using a smaller maximum segment size is that
`scm_i_adjust_min_yield ()' gets called more frequently (from
`scm_gc_for_newcell ()'), which explains the "smoother" increases of the
min yield.

Using the script I posted earlier, I get the following execution times
for 2500 iterations:

  * max-segment-size = 64 KiB

    real    3m1.167s
    user    2m56.200s
    sys     0m2.220s

  * max-segment-size = 20 MiB

    real    3m29.047s
    user    3m22.392s
    sys     0m4.764s

Thus it would be sensible to change `SCM_DEFAULT_MAX_SEGMENT_SIZE' to
some smaller value.

Thanks,
Ludovic.