Re: Finding Octave's memory use breakdown

[LachlanA]

John W. Eaton wrote
> On 04/24/2016 11:42 PM, LachlanA wrote:
>>
>> Is there a way to find out where Octave is using memory?  I often find
>> that
>> Octave is using say 2GB, but "whos" shows only a few hundred megabytes in
>> variables.
> 
> What are you looking at to determine that the overall memory usage is 
> 2GB?  If I start Octave (running the GUI and built from current 
> development sources with debug symbols) it looks to me like it takes 
> about 100 MB to run, with 80 MB of that being shared.  That's the 
> resident and shared memory sizes reported by top.  The total virtual 
> size is a little over 1 GB.  That doesn't seem terribly large.  Are you 
> concerned that it is?

I often deal with large data sets.  I just ran an example which gave

debug> whos
Variables in the current scope:

   Attr Name                  Size                     Bytes  Class
   ==== ====                  ====                     =====  =====
  a     argn                  4x4                         16  char
        capFactor1         3000x365x31             271560000  double
    f   cor_solar          3000x17520                  70084  double
    f   data               3000x365x48             210240000  single
        disconnected        365x3000                 1095000  logical
        first_solar_data      1x1                          8  double
        first_solar_day    3000x1                      24000  double
        gap                   1x1                          8  double
    f   ind                   1x3000                      24  double
        k                     1x1                          8  double
        last_solar_day     3000x1                      24000  double
    f   meta                  1x1                     164158  struct
        not_all               1x3000                    3000  logical
    f   s                     1x1                  655798080  struct
        s_su                  1x3000                   24000  double
        solar_correction   3000x365x31             271560000  double
        solar_user_range      1x3000                      24  double
    f   su                    1x3000                   24000  double
        u                     1x15                        60  double

Total is 174126036 elements using 1410586470 bytes



There was a similar amount in another couple of stack frames, though much of
it would point to the same "rep"s of octave_values.  After "dbquit; clear
all", top reports a virtual size of 2,685,748 bytes (down from 3.7G) and an
RSS of 1.032G (down from 2G), with 52792 shared.

It may be that some of that has gone back to the malloc/new pool but not
been returned to the OS, possibly because of fragmentation, in which case
the RSS would probably drop without performance penalty if another
application grabbed it.  However, I have no idea how much has been consumed
by the parse tree of the scripts I was running, or any other overheads.  I'm
guessing it is much less than 1GB, but that is just a guess.

If there is no way to find out where the memory is going, would there be
value in extending  whos  to report on memory use other than variables, such
as those I listed above?

(As an aside, I entirely agree that printf etc. don't need to be moved.  I
was just thinking that the only benefit of converting them to a .m file was
memory, which reminded me of this issue.)

Cheers,
Lachlan



--
View this message in context: 
http://octave.1599824.n4.nabble.com/Finding-Octave-s-memory-use-breakdown-tp4676477p4676506.html
Sent from the Octave - Maintainers mailing list archive at Nabble.com.