bug#43389: 28.0.50; Emacs memory leaks using hard disk all time

[Arthur Miller]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arthur Miller <arthur.miller@live.com>
>> Cc: Jean Louis <bugs@gnu.support>,  fweimer@redhat.com,
>>   43389@debbugs.gnu.org,  dj@redhat.com,  michael_heerdegen@web.de,
>>   trevor@trevorbentley.com,  carlos@redhat.com
>> Date: Mon, 23 Nov 2020 18:19:32 +0100
>> 
>> > The glibc malloc is the prime suspect anyway.  I don't really believe 
>> > Emacs had
>> > such a glaring memory leak.
>> 
>> This has to be something introduced fairly recently, right?
>
> Maybe, I'm not sure.  Since we introduced the pdumper, we use malloc
> somewhat differently, and OTOH glibc removed some of the malloc hooks
> we used to use in versions of Emacs before 26.  In addition, glibc is
> also being developed, and maybe some change there somehow triggered
> this.
It has past long since v 26, and pdumber as well :-) You know I am
rebuilding all the time and am on relatively latest master so I would
have noticed it earlier, so it must be something since last month or so,
I am not claiming anything exact, but not too far ago.

> As you see, there's more than one factor that could possibly be
> related.
Yeah; I understand that :-). 

>> I didn't have any such problems before, but since maybe few weeks ago, I
>> have also experienced heavy lockdowns of my entire OS. To the point
>> where entire X11 got unresposnsive, when it happens I can't even switch
>> to terminal to kill Emacs. What I do is Alt-Shift to another virtual
>> linux console. I don't even need to log into the system in that console,
>> I can then Alt-Shift 1 to go back to one I am logged into, and
>> everything is normal. Emacs is every time restarted by systemd and
>> everything is repsonsive and working as normal. 
>> 
>> This started sometime ago; and I have noticed that it happens when I was
>> cleaning my disk and reading big directories in Dired (I have some with
>> ~7k-10k files in them). I was using Helm to complete paths when I was
>> shifting fiels and folders around. After maybe hour or so I would
>> experience big slowdown. I don't have swap file on my system enabled at
>> all, so I am not sure what was going, but I didn't have time to
>> participate in this memory leak thing yet. I haven't experienced any
>> problems since I recompiled Emacs last time, which was in 18th (last
>> Wendesday). I have recompiled without Gtk this time, but I have no idea
>> if it has anything to do with the issue, was just a wild shot to see if
>> things are better.
>
> If the problem is memory, I suggest to look at the system log to see
> if there are any signs of that.
Nothing else crashes, and I have 32 gig, so I am not sure what can be a
problem.

It is obvious that Emacs causes the lockdown, but I don't know how.
I am not really sure what to make of the syslog in this case either.

You can take a peek  at the last crash I had (17th last week), if it
tells you anything more then what apps I use :-). I was playing music
with Emacs, so you will see start with pulseaudio, and what happened
untill Emacs restarted. As you see everything is happening in 4 seconds
interval, so it must be the point when I switched to another console
with Alt+Shift. I have no idea why systemd kills Emacs when I do that
either, but I discovered it does so. My intention from the beginning
was to just pkill Emacs, and hoped it was just X11 that was locked, not
entire system, but I discovered that I didn't even needed to kill emacs,
it was already killed by the time I logged into another console and
everything seemed to work nice after switch to other console, so I kept
using it as my workaround since it started; 3 - 4 weeks ago? At least
what I am aware of.

crash-log.txt
Description: Text document