igraph-help
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [igraph] graphs too large for RAM


From: Emmanuel Navarro
Subject: Re: [igraph] graphs too large for RAM
Date: Wed, 8 Jul 2009 10:25:58 +0200

Hello,

As said Tamas I developed a "SQL-based" version of igraph.
Thanks to the layered design of igraph, it's quite easy to rewrite the basic layer to make it use an other storage method.
If you need some help to install and get started with fell free to ask me.

As said Tamas we for seen to build a cache system, but it's not implemented yet.
It's a nice issue : what could be a good cache policy for a given algorithm ?

Emmanuel N.

On Fri, Jul 3, 2009 at 8:58 PM, Tamas Nepusz <address@hidden> wrote:
Hi Richard,


It doesn't look like igraph has support for the graph>RAM problem, though, but I thought I'd ask around to see how others deal with the problem (buying more RAM won't work for this machine).
We have a branch in igraph's Launchpad repository (maintained by Emmanuel Navarro) that replaces igraph's lowest in-memory graph storage layer with another one that stores the graph in an SQL database. Actually, there are two branches, one is based on the development tree, the other one is on the stable 0.5 tree:

https://code.launchpad.net/~enavarro222/igraph/0.5.1-db
https://code.launchpad.net/~enavarro222/igraph/0.6-db

I have never tried these branches myself, and I remember that Manu told that although it works, a caching layer on top the database layer would be desirable to make the thing efficient. There's another: some igraph functions start by converting whatever graph representation they get into an in-memory adjacency list, so at this point the graph must fit into the memory again (but of course one can rewrite these functions to use the original representation).

The relevant threads in the igraph-help mailing list archive are as follows (I think):

http://lists.gnu.org/archive/html/igraph-help/2009-03/msg00183.html
http://lists.gnu.org/archive/html/igraph-help/2008-12/msg00009.html
http://lists.gnu.org/archive/html/igraph-help/2008-12/msg00001.html

--
Tamas


_______________________________________________
igraph-help mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/igraph-help


reply via email to

[Prev in Thread] Current Thread [Next in Thread]