[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Findutils-patches] [PATCH] find memory leak
From: |
Bernhard Voelker |
Subject: |
Re: [Findutils-patches] [PATCH] find memory leak |
Date: |
Thu, 2 Feb 2017 23:31:07 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 |
Hi Goffredo,
On 02/02/2017 08:48 PM, Goffredo Baroncelli wrote:
> On 2017-02-02 00:34, Bernhard Voelker wrote:
>> On 12/15/2016 10:19 PM, Goffredo Baroncelli wrote:
>>> Hi All,
> [...]
>>>
>>> # Without the patch, this command requires up to 602MB of memory !!!
>>> $ /usr/bin/time ./find /lib/modules/ -ignore_readdir_race \( -fstype
>>> NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o
>>> -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o
>>> -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype
>>> devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o
>>> -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o
>>> -fstype ocfs2 -o -type d -false \) -prune -o -print0 | wc -l
>>> 31.59user 27.17system 0:58.82elapsed 99%CPU (0avgtext+0avgdata
>>> 602476maxresident)k
>>> 0inputs+0outputs (0major+150129minor)pagefaults 0swaps
>>>
>>> # With the patch, the same command as above requires only 2MB of memory !!!
>>> $ /usr/bin/time ./find /lib/modules/ -ignore_readdir_race \( -fstype
>>> NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o
>>> -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o
>>> -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype
>>> devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o
>>> -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o
>>> -fstype ocfs2 -o -type d -false \) -prune -o -print0 | wc -l
>>> 30.94user 26.14system 0:57.14elapsed 99%CPU (0avgtext+0avgdata
>>> 2764maxresident)k
>>> 0inputs+0outputs (0major+165minor)pagefaults 0swaps
>>>
>>>
>>> # for curiosity my /lib/modules contains
>>> $ find /lib/modules/ | wc -l
>>> 68101
>>>
>>> Finally, what about caching the value of read_file_system_list() instead of
>>> recomputing it every time ? On the basis of my tests I found an impressive
>>> gain in terms of cpu usage. But what are the cons ?
>>>
>>>
>>> BR
>>> G.Baroncelli
>>
>> I cannot reproduce here - i.e. I do not see such a huge memory leak here ...
>> even
>> if I add some 8 bind-mounts so that /proc/self/mountinfo has 30k.
>> How does your test system look like? Is there anything special about your
>> directory
>> structure?
>
> I don't think that my directory structure is special in any way.
>
> $ ls /lib/mnodules
> 3.16.0-4-amd64 4.3.0-trunk-amd64 4.6.0-1-amd64 4.8.15 4.9.5
> 3.16-3-amd64 4.4.0-rc8-amd64 4.8.0-2-amd64 4.9.3
>
>
> In your testa, have you used all the "-fstype ...." arguments ? IIRC the leak
> size depends by the number of "-fstype ...."
yes, I did. But my /lib/modules only has 9470 files, so the memory usage
without
the patches was only
Maximum resident set size (kbytes): 4136
and with the patches it's down to
Maximum resident set size (kbytes): 2556
So I'm wondering why your find uses 110M. Even if I try for '/' (which has
1281671 files output by that find command), then only 16-18M (without/with the
patch)
are used here:
Maximum resident set size (kbytes): 17892
Maximum resident set size (kbytes): 15800
Thus said, 110M sound excessive.
>> What does 'valgrind --leak-check=full --show-leak-kinds=all ./find ...'
>> tell you about the leak(s)? How many lost blocks do you have?
>
> Below the output:
>
>
>
> $ valgrind --leak-check=full --show-leak-kinds=all /usr/bin/find
> /lib/modules/ -ignore_readdir_race \( -fstype NFS -o -fstype nfs -o
> -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype
> smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype ncpfs -o -fstype coda
> -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o -fstype mfs -o -fstype
> shfs -o -fstype sysfs -o -fstype cifs -o -fstype lustre_lite -o -fstype tmpfs
> -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 -o -type d -false \)
> -prune -o -print0 | wc -l
> ==4520== Memcheck, a memory error detector
>
> ==4520== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
>
> ==4520== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
>
> ==4520== Command: /usr/bin/find /lib/modules/ -ignore_readdir_race ( -fstype
> NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o
> -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype
> ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o
> -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype
> lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2
> -o -type d -false ) -prune -o -print0
> ==4520==
>
>
>
> 0
> ==4520==
> ==4520== HEAP SUMMARY:
> ==4520== in use at exit: 7,625,040 bytes in 3,340,960 blocks
> ==4520== total heap usage: 17,220,046 allocs, 13,879,086 frees, 550,821,122
> bytes allocated
> ==4520==
> ==4520== 8 bytes in 1 blocks are still reachable in loss record 1 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x117AE8: ??? (in /usr/bin/find)
> ==4520== by 0x1106E2: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x110CDD: pred_and (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1111BD: pred_or (in /usr/bin/find)
> ==4520==
> ==4520== 16 bytes in 1 blocks are still reachable in loss record 2 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x115A12: ??? (in /usr/bin/find)
> ==4520== by 0x10F0BF: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 56 bytes in 1 blocks are still reachable in loss record 3 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x126EAC: ??? (in /usr/bin/find)
> ==4520== by 0x118DF9: ??? (in /usr/bin/find)
> ==4520== by 0x119003: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 4 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x11A614: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 5 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112E04: ??? (in /usr/bin/find)
> ==4520== by 0x119E7A: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 6 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x119E7A: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 7 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x1146C3: ??? (in /usr/bin/find)
> ==4520== by 0x11804C: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 8 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112E04: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x11BCD7: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 9 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x11BCD7: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 10 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112D68: ??? (in /usr/bin/find)
> ==4520== by 0x119C48: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 11 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112E04: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x117ED8: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 12 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x117ED8: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 13 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x118FBD: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 14 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112475: ??? (in /usr/bin/find)
> ==4520== by 0x11359A: ??? (in /usr/bin/find)
> ==4520== by 0x114027: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 15 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112475: ??? (in /usr/bin/find)
> ==4520== by 0x113649: ??? (in /usr/bin/find)
> ==4520== by 0x114027: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 16 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112475: ??? (in /usr/bin/find)
> ==4520== by 0x11359A: ??? (in /usr/bin/find)
> ==4520== by 0x11360A: ??? (in /usr/bin/find)
> ==4520== by 0x11360A: ??? (in /usr/bin/find)
> ==4520== by 0x114027: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 312 bytes in 1 blocks are still reachable in loss record 17 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112475: ??? (in /usr/bin/find)
> ==4520== by 0x11359A: ??? (in /usr/bin/find)
> ==4520== by 0x11360A: ??? (in /usr/bin/find)
> ==4520== by 0x11360A: ??? (in /usr/bin/find)
> ==4520== by 0x11360A: ??? (in /usr/bin/find)
> ==4520== by 0x114027: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 616 bytes in 273 blocks are definitely lost in loss record 18 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BE9A: ??? (in /usr/bin/find)
> ==4520== by 0x117A1B: ??? (in /usr/bin/find)
> ==4520== by 0x11BE2E: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 1,936 bytes in 858 blocks are indirectly lost in loss record 19 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BE9A: ??? (in /usr/bin/find)
> ==4520== by 0x11BD68: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 2,184 bytes in 7 blocks are still reachable in loss record 20 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x11BDC4: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 4,992 bytes in 16 blocks are still reachable in loss record 21 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112DBE: ??? (in /usr/bin/find)
> ==4520== by 0x114706: ??? (in /usr/bin/find)
> ==4520== by 0x11BCD7: ??? (in /usr/bin/find)
> ==4520== by 0x11BDA2: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 6,424 bytes in 858 blocks are indirectly lost in loss record 22 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BEAB: ??? (in /usr/bin/find)
> ==4520== by 0x11BD68: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 6,688 bytes in 858 blocks are indirectly lost in loss record 23 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BE75: ??? (in /usr/bin/find)
> ==4520== by 0x11BD68: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 7,488 bytes in 24 blocks are still reachable in loss record 24 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127F98: ??? (in /usr/bin/find)
> ==4520== by 0x112C90: ??? (in /usr/bin/find)
> ==4520== by 0x112D68: ??? (in /usr/bin/find)
> ==4520== by 0x119B88: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 14,080 bytes in 858 blocks are indirectly lost in loss record 25 of
> 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BE87: ??? (in /usr/bin/find)
> ==4520== by 0x11BD68: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 46,816 bytes in 836 blocks are indirectly lost in loss record 26 of
> 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x12BE65: ??? (in /usr/bin/find)
> ==4520== by 0x11BD68: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 77,176 (1,232 direct, 75,944 indirect) bytes in 22 blocks are
> definitely lost in loss record 27 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x12BE65: ??? (in /usr/bin/find)
> ==4520== by 0x11BD68: ??? (in /usr/bin/find)
> ==4520== by 0x113EC7: ??? (in /usr/bin/find)
> ==4520== by 0x10F158: ??? (in /usr/bin/find)
> ==4520== by 0x53842B0: (below main) (libc-start.c:291)
> ==4520==
> ==4520== 1,075,448 bytes in 476,619 blocks are definitely lost in loss record
> 28 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BE9A: ??? (in /usr/bin/find)
> ==4520== by 0x117A1B: ??? (in /usr/bin/find)
> ==4520== by 0x1106E2: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1111BD: pred_or (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x110CDD: pred_and (in /usr/bin/find)
> ==4520==
> ==4520== 1,075,448 bytes in 476,619 blocks are definitely lost in loss record
> 29 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BE9A: ??? (in /usr/bin/find)
> ==4520== by 0x117A1B: ??? (in /usr/bin/find)
> ==4520== by 0x1106E2: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x110CDD: pred_and (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520==
> ==4520== 5,377,240 bytes in 2,383,095 blocks are definitely lost in loss
> record 30 of 30
> ==4520== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
> ==4520== by 0x127E28: ??? (in /usr/bin/find)
> ==4520== by 0x127FE3: ??? (in /usr/bin/find)
> ==4520== by 0x12BE9A: ??? (in /usr/bin/find)
> ==4520== by 0x117A1B: ??? (in /usr/bin/find)
> ==4520== by 0x1106E2: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1111BD: pred_or (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520== by 0x1111BD: pred_or (in /usr/bin/find)
> ==4520== by 0x1158A8: ??? (in /usr/bin/find)
> ==4520==
> ==4520== LEAK SUMMARY:
> ==4520== definitely lost: 7,529,984 bytes in 3,336,628 blocks
> ==4520== indirectly lost: 75,944 bytes in 4,268 blocks
> ==4520== possibly lost: 0 bytes in 0 blocks
> ==4520== still reachable: 19,112 bytes in 64 blocks
> ==4520== suppressed: 0 bytes in 0 blocks
> ==4520==
> ==4520== For counts of detected and suppressed errors, rerun with: -v
> ==4520== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0)
>
>
> Pay attention that I removed some kernel modules directories from
> /lib/modules after my email, so now the memory usage is lower
>
> /usr/bin/time /usr/bin/find /lib/modules/ -ignore_readdir_race \(
> -fstype NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype
> binfmt_misc -o -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype
> iso9660 -o -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs
> -o -fstype devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype
> cifs -o -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf
> -o -fstype ocfs2 -o -type d -false \) -prune -o -print0 | wc -l
> 5.62user 5.00system 0:10.63elapsed 99%CPU (0avgtext+0avgdata
> 110204maxresident)k
> 0inputs+0outputs (0major+26952minor)pagefaults 0swaps
>
> (only 110MB used, but before the memory usage was ~600MB)
Did you try with your patch (or incl. my patches) as well?
The above valgrind output doesn't look too bad, actually. It seems
that it is not (only) a memory leak, but instead regular use of memory which
is correctly free()d: only those ~8M are lost (and parts of that would be freed
with the patches).
Have a nice day,
Berny
- Re: [Findutils-patches] [PATCH] find memory leak, Bernhard Voelker, 2017/02/01
- Re: [Findutils-patches] [PATCH] find memory leak, Goffredo Baroncelli, 2017/02/02
- Re: [Findutils-patches] [PATCH] find memory leak,
Bernhard Voelker <=
- Re: [Findutils-patches] [PATCH] find memory leak, Goffredo Baroncelli, 2017/02/03
- Re: [Findutils-patches] [PATCH] find memory leak, Bernhard Voelker, 2017/02/05
- Re: [Findutils-patches] [PATCH] find memory leak, Goffredo Baroncelli, 2017/02/06
- Re: [Findutils-patches] [PATCH] find memory leak, Bernhard Voelker, 2017/02/06
- Re: [Findutils-patches] [PATCH] find memory leak, Goffredo Baroncelli, 2017/02/07
- Re: [Findutils-patches] [PATCH] find memory leak, Bernhard Voelker, 2017/02/07
- Re: [Findutils-patches] [PATCH] find memory leak, Dale R. Worley, 2017/02/08
- Re: [Findutils-patches] [PATCH] find memory leak, Bernhard Voelker, 2017/02/08
- Re: [Findutils-patches] [PATCH] find memory leak, Goffredo Baroncelli, 2017/02/08
- Re: [Findutils-patches] [PATCH] find memory leak, Bernhard Voelker, 2017/02/08