[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files'

From: Michael Heerdegen
Subject: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files'
Date: Wed, 28 Feb 2018 11:58:49 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)

Noam Postavsky <address@hidden> writes:

> >> #+begin_src emacs-lisp
> >> (seq-doseq (_ (stream-range 1 1000000)) nil)
> >> #+end_src
> >> 
> >> Note that this is executed as a loop due how to streams are
> >> implemented, although the definition of `seq-doseq' looks
> >> recursive.
> Doesn't look recursive to me, it expands to a call to seq-do, which uses
> a simple loop.

I was imprecise, I meant that the streams are defined recursively (most
of the time).  Though, it's a delayed type of recursion.  Anyway, I
think that this doesn't matter here.

> > Does anyone has a reasonable idea for avoiding the crash in such
> > programs?
> I don't have a quick answer for the general case, but I think it's a bug
> in stream.el that it's creating such large structures in the first
> place.  As far as I understand it, the point of streams is to handle
> long lists by encoding them as

Yes, that's exactly how it's implemented.  When requesting more elements
from the stream, that becomes

      (FIRST-VALUE .

When we loop over the string, the cons whose car is the FIRST-VALUE,
let's call it cons1, is immediately thrown away, and we continue with



AFAIU the problem is that the cons1 still exists in memory until garbage
collection kicks in.  When that happens, the cons1 points to a largely
recursive cons structure, though this structure is never referenced from
Lisp in this form.
> so as to avoid allocating large amounts of memory.  Is there an easy way
> to find out what the large structures are, and where they are coming
> from?

I think I've answered that.  At least, I think so.  What I don't
understand is that when I force the `seq-doseq' to call
`garbace-collect' explicitly every 1000 cycles, or so, it doesn't help:
the crash still happens after generating ~ 70 000 elements, or some
more, but I can't avoid the crash, no matter how often I call gc.  So
I'm not sure whether these long lists are the problem or something else.
The FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST looks harmless when I print
it, even after thousands of iterations, so I would not understand why
that could be problematic.  streams.el implements things in a way that
these rest functions are not deeply nested lambdas.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]