[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug-recutils] Optimizing memory consumption

From: Michał Masłowski
Subject: Re: [bug-recutils] Optimizing memory consumption
Date: Wed, 15 Aug 2012 15:08:49 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/24.1 (gnu/linux)


> I used Valgrind's massif tool in order to profile the usage of the heap
> by recsel, and got the following results running recsel on a file
> containing 20000 records (3.8 Mb):

Not used this tool before, will be useful.

> There are no memory leaks.  As you can see, the peak indicates a total
> of memory used of around 121Mb in the heap.  This is simply too much for
> a 4 megabytes recfile, so we must come with some optimizations in terms
> of memory consuption.

There are some leaks with othoer options (at least with some indexing
changes), planning to find and fix more after the index support is
complete.  I don't expect this to have significant effects.

> For example, an obvious optimization is to use a pool of strings to
> store the field names.  The pool would be shared by all the records
> pertaining to some record set.

Field sources might be more commonly duplicated.  Similarly for values
of some fields.

Seen many places in librec doing deep copies of objects, this would
probably need reference counting or a more complex solution to use
references to the same objects instead.

Attachment: pgpPdzZ2TLeJn.pgp
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]