Re: Memory usage report

[Ihor Radchenko]

> Did you actually see cases where something like that happens?  I mean
> that loading some library, or all of your Org files, causes increase
> in memory usage of hundreds of MBs (320MB in your example)?  I only
> see this when the buffers themselves are near that value.

Yep. I see it every time I run Emacs. I have done M-x memory-usage
before and after loading org-agenda (which loads all my working org
files).

Before:
Total in lisp objects: 77.4MB (live 47.7MB, dead 29.7MB)
Buffer ralloc memory usage:
16 buffers
39.3kB total (23.6kB in gaps)

After:
Total in lisp objects:  283MB (live  218MB, dead 64.7MB)
Buffer ralloc memory usage:
47 buffers
6.79MB total (78.6kB in gaps)

This is not all the memory increase. Some part seems to be in heap.

Also, the memory usage is even worse when org-mode is using overlays (I
am on org-mode branch using text properties instead of overlays in org).

Full memory-usage reports are shown below.

The memory consumption gets even worse after I use Emacs for several
hours. Recently, it tend to settle around 20% (of 8Gb) with lisp objects
taking around 1Gb.

This last part is the most annoying and also difficult to track - I have
no easy way to know if it is also caused by org-mode or it is something
else.

Best,
Ihor

Before:

Garbage collection stats:
((conses 16 934007 1096370) (symbols 48 73755 2041) (strings 32 265462 116861) 
(string-bytes 1 9693220) (vectors 16 115429) (vector-slots 8 1423224 1215629) 
(floats 8 1419 3034) (intervals 56 1366 249) (buffers 992 16))

 =>     14.3MB (+ 16.7MB dead) in conses
        3.38MB (+ 95.7kB dead) in symbols
        8.10MB (+ 3.57MB dead) in strings
        9.24MB in string-bytes
        1.76MB in vectors
        10.9MB (+ 9.27MB dead) in vector-slots
        11.1kB (+ 23.7kB dead) in floats
        74.7kB (+ 13.6kB dead) in intervals
        15.5kB in buffers

Total in lisp objects: 77.4MB (live 47.7MB, dead 29.7MB)

Buffer ralloc memory usage:
16 buffers
39.3kB total (23.6kB in gaps)
      Size      Gap     Name

     13213      2000    *scratch*
      1252      769     *Messages*
       676      1574    *Buffer Details*
       294      1940     *straight-watcher*
       265      1756    *straight-process*
       124      1911     *code-conversion-work*
       112      3941    *helm M-x*
        82      2000    *elfeed-log*
        67      2000     *pdf-info-query--parse-response*
        53      2000     *Echo Area 1*
        24      2001     *Echo Area 0*
         8      12       *pdf-info-query--escape*
         0      20       *Minibuf-1*
         0      20       *Minibuf-0*
         0      20       *epdfinfo*
         0      2067     tq-temp-epdfinfo

After:

Garbage collection stats:
((conses 16 9393579 2919763) (symbols 48 75965 293) (strings 32 323708 224589) 
(string-bytes 1 13017637) (vectors 16 154787) (vector-slots 8 2249055 1365412) 
(floats 8 1630 2854) (intervals 56 547977 53455) (buffers 992 47))

 =>      143MB (+ 44.6MB dead) in conses
        3.48MB (+ 13.7kB dead) in symbols
        9.88MB (+ 6.85MB dead) in strings
        12.4MB in string-bytes
        2.36MB in vectors
        17.2MB (+ 10.4MB dead) in vector-slots
        12.7kB (+ 22.3kB dead) in floats
        29.3MB (+ 2.85MB dead) in intervals
        45.5kB in buffers

Total in lisp objects:  283MB (live  218MB, dead 64.7MB)

Buffer ralloc memory usage:
47 buffers
6.79MB total (78.6kB in gaps)
      Size      Gap     Name

   2345866      2000    nodeadline.org
   1914956      2000    notes.org
    695164      2000    TODO.org
    548214      2000    config.org
    487501      2000    get_started_org_mode.org
    361666      2000    CuNb-beamcrack.org
    340586      2000    References.org
     84150      2000    articles.org
     47507      2000    Journal.org
     45875      2000    Олди Генри Лайон.org
     31561      2000    mirror-text.org
     30235      2000    rss.org
     29662      2000    org-autosort.org
     18633      2000    [book]DFR.org
     14412      801     *scratch*
      9932      2000    schedule.org
      8719      2000    Abraham Daniel.org
      7292      2000    org-mode.org
      6172      2000    inbox.org
      4503      2000    contacts.org
      2564      2000     *code-conversion-work*
      1944      2000    gnuplot.org
      1838      183     *Messages*
      1125      950     *Memory-Profiler-Report 2020-09-18 23:59:40*
       897      1131     *emacsql-log*
       889      1727     *Agenda Commands*
       681      1569    *Buffer Details*
       301      1720    *Org Agenda(d:)*
       294      1940     *straight-watcher*
       265      1756    *straight-process*
       125      3928    *helm M-x*
       119      2000    frommobile.org
        82      2000    *elfeed-log*
        67      2000     *pdf-info-query--parse-response*
        50      1973    *Helm Completions*
        41      1988    *helm-mode-profiler-report-write-profile*
        35      1990     *Echo Area 0*
        31      1991    *helm-mode-profiler-start*
        13      2474     *emacsql-sqlite*
         8      12       *pdf-info-query--escape*
         6      14       *temp*
         0      2030     *Minibuf-1*
         0      20       *Minibuf-0*
         0      2053     *Echo Area 1*
         0      20       *epdfinfo*
         0      2067     tq-temp-epdfinfo
         0      20       *server*



Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@gmail.com>
>> Cc: larsi@gnus.org, emacs-devel@gnu.org
>> Date: Fri, 18 Sep 2020 22:30:11 +0800
>> 
>> Seeing how much memory is used by buffer-local variables in org
>> buffers would help. Similarly, total memory consumption in variables
>> from individual packages would be helpful to pinpoint why the memory
>> usage is high.
>
> That'd require some filtering of the variables, not just finding the
> largest one.
>
> Did you actually see cases where something like that happens?  I mean
> that loading some library, or all of your Org files, causes increase
> in memory usage of hundreds of MBs (320MB in your example)?  I only
> see this when the buffers themselves are near that value.