Re: NSAutoreleasePool does not free all memory

[Stefan Boehringer]

Hello all,

IMO it is no good programming style (though sometimes unavoidable) to change higher level code to accomodate for insufficiencies in lower level code.

It is more a problem of the kernel (linux/bsd) which prevents swapping in this case. If the kernel offered VM remapping the malloc/free implementation could unmap any complete pages which become free by memory release anywhere *within* your process memory.
Linux, however, doesn't offer VM remapping. The real solution would be to move to a proper kernel (Mach etc.) which offers flexible VM handling and to change the library functions accordingly (as Mem/$ is still growing exponentially such optimizations become a bit academical for real life apps (as apposed to some scientific ones) - ever wondered why Win/Mac have become usable today? :-).
Best wishes,

	Stefan

Richard Frith-Macdonald wrote:
> On Monday, April 30, 2001, at 02:37 PM, David Wetzel wrote:
> 
> > Richard Frith-Macdonald wrote
> >
> >> Your problem is that you are using a relatively meaningless
> >> statistic ... the reported memory
> >> size of an application is actually its *peak* size, not its current
> >> size.  True, it's relevant
> >> to the swap space on your system, but not much else.
> >
> > I understood that. You have some kind of "internal" free...
> 
> Not sure what you mean by that.  free() is part of the libc library - I 
> guess
> you could call it 'internal' since, other than calling brk() or sbrk() 
> it has
> nothing tom do with the memory management provided by the operating 
> system.
> 
> >> In fact, your little test program demonstrates an unrealistic and
> >> misleading situation when
> >> not using autorelease pools, and a more normal situation when using 
> >> them.
> >>
> >> I could change the autorelease pool implementation so that pools are 
> >> not
> >> cached and the memory
> >> they use is freed as soon as the pool is released, but the only effect
> >> that would have on real
> >> programs is to slow them down slightly as each creation/destruction 
> >> of a
> >> pool would
> >> require a malloc/free overhead.
> >
> > My guess is that this malloc/free overhead is much less (in time and 
> > CPU usage)
> > than the overhead that we have when we are paging memory to disk.
> 
> But the operating system won't page memory to/from disk unless it has 
> run out
> of real memory (or has nothing better to do with its time) in which case,
> the operating system will have to do paging/swapping to support a 
> malloc() call
> anyway.  So in the situations where paging is occuring, malloc/free 
> causes
> paging *in addition to* its other overheads.
> 
> > The best way to do this would be a #ifdef SYSTEM_HAS_EFFICIENT_MALLOC 
> > in the Autorelease Pool source.
> > If my system's malloc would be inefficient, I could undef this #define.
> 
> 1. it's not really an issue of mallocs efficiency/inefficiency ... 
> however efficient it is
> made in practice, it will always be more efficient to avoid a 
> malloc/free if possible.
> 
> 2. In a real program it would generally make no difference anyway ... 
> other allocated objects
> would stop memory being returned to the operating system.  So making the 
> autorelease pool
> code refrain from caching would not actually help in any way - the total 
> program data size
> would still be large.
> 
> 3. Even in a highly artificial program, where a large amount of memory 
> is used and then
> memory usage drops back to a small fraction of that, and the 
> program/libraries are
> carefully hacked to ensure that the memory is given back to the 
> operating system, all
> you would gain is a lower swapspace usage.
> 
> 
> In conclusion ... this is the way the unix/linux virtual memory system 
> is *designed*
> to work.  It makes no sense to try to do things differently except in 
> highly specialised
> circumstances.
> 
> _______________________________________________
> Discuss-gnustep mailing list
> Discuss-gnustep@gnu.org
> `http://mail.gnu.org/mailman/listinfo/discuss-gnustep
>