Re: How to avoid large unwanted cell-heap-segments?

[Roland Orre]

I solved my gc-problem with a quick and dirty solution so far.
With the procedure below "gc-heap-size" I can set the heap size
to a low value when my data storage is allocated. 
By doing:
(define gc-heap1 (gc-heap-size 1))
(define gc-heap2 (gc-heap-size 2))
and then doing (gc-heap-size 1 gc-heap1))
at some critical points my process now runs smoothly without
allocating incredible much cell heap.
(the reason for doing this is that I was stubborn enough to
 stick to running this project on my laptop which only has 1Gb,
 but I saw that as no problem as the data base only used 300 Mb)

It is now 6 years since the generational gc discussion started
on this list. Some time in the future I may be interested in
implementing it, but.. I have a lot of other stuff that need
implementation first (as I think for most of us) but I guess
it is easier now with the garbage bits separated. Has anyone
had any recent consideration about generational gc?

Here is a link I found:
http://www.ipsj.or.jp/members/Trans/Eng/03/2000/4109/article008.html
and here are some comments from 2000 from the guile list
http://sources.redhat.com/ml/guile/2000-03/msg00561.html

        Best regards
        Roland Orre


extern struct scm_t_cell_type_statistics scm_i_master_freelist;
extern struct scm_t_cell_type_statistics scm_i_master_freelist2;

SCM_PROC(s_gc_heap_size, "gc-heap-size", 1, 1, 0, scm_gc_heap_size);
SCM scm_gc_heap_size( SCM which, SCM hsize )
{
  long newsize;
  SCM_ASSERT( SCM_INUMP(which), which, SCM_ARG1, s_gc_heap_size );
  if (! SCM_UNBNDP(hsize)) {
    SCM_ASSERT( SCM_INUMP(hsize), hsize, SCM_ARG2 ,s_gc_heap_size);
    newsize=SCM_INUM(hsize);
  }
  switch (SCM_INUM(which)) {
  case 1:
    if (!SCM_UNBNDP(hsize))
      scm_i_master_freelist.heap_size = newsize;
    return SCM_MAKINUM(scm_i_master_freelist.heap_size);
    break;
  case 2:
    if (!SCM_UNBNDP(hsize))
      scm_i_master_freelist2.heap_size = newsize;
    return SCM_MAKINUM( scm_i_master_freelist2.heap_size );
    break;
  default:
    scm_wta( hsize, (char *)SCM_ARG2, s_gc_heap_size );
  }
} /* scm_gc_heap_size */



On Mon, 2004-10-11 at 14:33, Roland Orre wrote:
> Does anyone have any hints about how to avoid large unwanted
> cell-heap-segments?
> 
> Between these two statistics below, the memory usage has increased
> from 460 MB to 640 MB, the only thing being done is performing an
> iteration over a vector (3109805 items) and performing a few
> array-map! and some float calculations for each vector item.
> 
> The only noticable difference are these cell-heap-segments:
> (1394960384 . 1575913472) (1578743808 . 1715167232)
> 
> After this each garbage collection run takes a lot more time...
> If generational gc was implemented then one solution would possibly
> be to dedicate a "generation" for static memory.
> 
> Ideally I would now want to be able to allocate my data structures
> (cells, vectors and uniform vectors) in non gc visible memory, to
> not let the garbage collector bother at all about this large static
> memory (a data base read from disc). (I could of course do this by
> using private structures and smobs, but this implies more work...)
> 
> Any hint how I could (easy...) avoid this kind of problem?
> About 8 years ago I had a similar problem, then I patched gc.c
> and changed the cell heap allocation memory growth.
> 
>       Best regards
>       Roland Orre
> 
> ;before
> ((gc-time-taken . 48862) (cells-allocated . 33781163)
>  (cell-heap-size . 33994558) (bytes-malloced . 172137454)
>  (gc-malloc-threshold . 179237085) (gc-times . 318)
>  (gc-mark-time-taken . 48541) (cells-marked . 1690839800)
>  (cells-swept . 3342457125) (malloc-yield . 2) (cell-yield . 42)
>  (cell-heap-segments
>   (134795264 . 134830080) (1077979136 . 1078243328)
>   <snip>
>   (1241004032 . 1301637120) (1302587392 . 1393537024)))
> 
> ;after
> ((gc-time-taken . 52196) (cells-allocated . 38980360)
>  (cell-heap-size . 73511652) (bytes-malloced . 177976821)
>  (gc-malloc-threshold . 179237085) (gc-times . 322)
>  (gc-mark-time-taken . 51871) (cells-marked . 1837245337)
>  (cells-swept . 3545722215) (malloc-yield . 2) (cell-yield . 29)
>  (cell-heap-segments
>   (134795264 . 134830080) (1077979136 . 1078243328)
>   <snip>
>   (1241004032 . 1301637120) (1302587392 . 1393537024)
>   (1394960384 . 1575913472) (1578743808 . 1715167232)))
> 
> 
> 
> 
> _______________________________________________
> Guile-user mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/guile-user