Re: [igraph] Memory usage

[Tamas Nepusz]

> Also, because it's necessary to check that a substring doesn't already
> exist in the graph, I created a vertex attribute containing the
> substring. I assumed that this is what is causing the huge memory
> footprint, but 6.3k seems a little extreme (the strings are only 32
> characters long).
Vertex attributes are stored in a dictionary that is attached to the graph via 
a hidden pointer, so the only overhead of using attributes (apart from the 
attributes themselves of course) is that extra dict. Theoretically, it 
shouldn't weigh 6.3 kbytes. It might happen that there's a memory (or reference 
count) leak somewhere in the Python interface, so if you could send me your 
code, I'll try to take a closer look at it.

> Also, I'm not sure what you mean about the Python interpreter's tendency
> (or not) to return memory to the OS
I have meant what's written in this blog entry:

http://evanjones.ca/python-memory.html

and in this mailing list thread:

http://mail.python.org/pipermail/python-list/2004-October/283092.html

Actually, since these are from 2004, there is a chance that things have changed 
in the Python interpreter since then. Plus, the case is even more complicated 
by that; for instance, when the Python garbage collector determines that an 
igraph Graph object can be deleted, it calls its destructor, which, in turn, 
does release memory back to the OS since the memory taken up by the graph 
itself was not allocated via Python's memory allocator anyway.

The bottom line is that it might have happened that you've found a reference 
count leak in the Python interface -- I agree with you that 6.3K is too much 
for storing a single vertex. If you can, please send your code to my private 
email address and I'll check what's going on.

-- 
Tamas