bug#38345: 27.0.50; Permanent increase in memory consumption after opening images (or pdfs)

[Ihor Radchenko]

> It may be useful to try memory mapping tools that show which part(s)
> of a program "own" specific regions of memory.  It could be that the
> memory growth is due to some library Emacs uses to show images, for
> example.  Those tools usually also show the memory in the free pool of
> the program's address space, so you should be able to see what's going
> on when you call clear-image-cache.

I tried to follow EmacsWiki
(https://www.emacswiki.org/emacs/EmacsMemoryDebugging).

The largest entry in smaps:

1. Before opening images

562e4cf38000-562e4d395000 rw-p 00000000 00:00 0                          [heap]
Size:               4468 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                4148 kB
Pss:                4148 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:      4148 kB
Referenced:         4148 kB
Anonymous:          4148 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB
THPeligible:            0
VmFlags: rd wr mr mw me ac

2. After opening images with all the buffers open

562e4cf38000-562e945a5000 rw-p 00000000 00:00 0                          [heap]
Size:            1169844 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:             1169500 kB
Pss:             1169500 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:   1169500 kB
Referenced:      1169500 kB
Anonymous:       1169500 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB
THPeligible:            0
VmFlags: rd wr mr mw me ac

3. After killing all the image buffers

562e4cf38000-562e945a5000 rw-p 00000000 00:00 0                          [heap]
Size:            1169844 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:             1169692 kB
Pss:             1169692 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:   1169692 kB
Referenced:      1169692 kB
Anonymous:       1169692 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB
THPeligible:            0
VmFlags: rd wr mr mw me ac

4. After calling (clear-image-cache t) and (garbage-collect)

562e4cf38000-562e945a5000 rw-p 00000000 00:00 0                          [heap]
Size:            1169844 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:             1169692 kB
Pss:             1169692 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:   1169692 kB
Referenced:      1169692 kB
Anonymous:       1169692 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB
THPeligible:            0
VmFlags: rd wr mr mw me ac 

The full info from smaps and output from pmap are attached:
*0.txt ::  Before opening images
*open.txt :: With all the image buffers
*kill.txt :: After killing all the image buffers
*clear.txt :: After clearing cache and garbage collecting

I also attempted to use perf, but I cannot understand much of
information from there and I was not able to make the scripts I use work
with the output. If you need, I can also generate perf output (may take
some time).

Regards,
Ihor

pmap0.txt
Description: Text document

pmap-open.txt
Description: Text document

pmap-kill.txt
Description: Text document

pmap-clear.txt
Description: Text document

smaps0.txt
Description: Text document

smaps-open.txt
Description: Text document

smaps-kill.txt
Description: Text document

smaps-clear.txt
Description: Text document

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@gmail.com>
>> Cc: 38345@debbugs.gnu.org
>> Date: Sun, 24 Nov 2019 00:04:17 +0800
>> 
>> Hmm. Following an article by Dima Kogan [1], it appears to me that free()
>> should de-allocate memory in the program's address space (or at least
>> memory drop should be visible in the plot I got, since I used the same
>> memory debugging tools)
>
> It may be useful to try memory mapping tools that show which part(s)
> of a program "own" specific regions of memory.  It could be that the
> memory growth is due to some library Emacs uses to show images, for
> example.  Those tools usually also show the memory in the free pool of
> the program's address space, so you should be able to see what's going
> on when you call clear-image-cache.