bug#17535: 24.3.91; Problems with profiling memory

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> 1. Visit profiler.el and go to this function:
>
>     (defun profiler-cpu-profile ()
>       "Return CPU profile."
>       (when (profiler-running-p 'cpu)
>       (profiler-make-profile
>        :type 'cpu
>        :timestamp (current-time)
>        :log (profiler-cpu-log))))
>
>    Position cursor on "profiler-make-profile" and type "C-h f".  Emacs
>    pops up a *Help* buffer saying:
>
>      profiler-make-profile is a compiled Lisp function in `profiler.el'.
>
>      (profiler-make-profile &key TAG VERSION TYPE LOG TIMESTAMP DIFF-P)
>
>      This function has a compiler macro `profiler-make-profile--cmacro'.
>
>      Not documented.
>
>    Click on the "profiler.el" link in the *Help* buffer -- Emacs
>    displays an error message saying "Unable to find location in file"

This now seems like it's fixed.

> 3. Displaying a memory profile after several minutes shows clear signs
>    of overflow:
>
>   - command-execute                                         151,652,741   0%
>    - call-interactively                                     151,650,661   0%
>     - dired-x-find-file                                      44,210,847   0%
>      - find-file                                             44,210,847   0%
>       - find-file-noselect                                   44,209,884   0%
>   ...
>       - normal-backup-enable-predicate                      7,091,148  -1%

I'm not able to reproduce this on the current trunk.

Eli Zaretskii <eliz@gnu.org> writes:

>> Other reason is that deallocation largely takes place during GC and it's
>> unclear to which functions/backtraces to attribute the corresponding
>> negative (attributing them to the backtrace that happens to be current
>> during GC would make the numbers pretty arbitrary).
>
> First, there are quite a few places where we allocate and deallocate
> memory for things other than Lisp objects.  Normally, such memory is
> deallocated right away, or very soon, after it is no longer needed.
> In those cases, we should attribute the memory freeing to the current
> function/backtrace.
>
> And second, I understand the limitations caused by GC, but I think we
> should still attribute memory freed during GC to GC itself.  That way,
> at least at some high enough level the positive and negative probes
> will balance, and the memory profile won't look like one giant leak.
>
> If we don't count free as a negative allocation, can you envision a
> situation for which the memory profile, as we have it now, would be a
> useful tool?  Because I can't.

If we started to try to count how much was freed, too, then the numbers
would become more arbitrary?  Because GC will free things
asynchronously, etc.  Just counting the allocations sounds like a valid
thing to do to me.

But perhaps counting frees would be nice as an option?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no