Re: using finalizers

[Eli Zaretskii]

> From: Tomas Hlavaty <tom@logand.com>
> Cc: rudi@constantly.at, emacs-devel@gnu.org
> Date: Sun, 02 Jan 2022 14:10:02 +0100
> 
> - Emacs Lisp should provide open-directory, read-directory and
>   close-directory functions.

If real-life use cases are brought up that cannot be adequately
handled by the existing APIs, we will consider extending them or
adding new APIs.  But the use cases need to come first.

> > I don't see why.  A callback can easily insert files into the buffer,
> > and trigger redisplay either directly or via timers and such.
> 
> But the control is in the hand of the mapping function.

No, the callback can do that under its own control.

> Triggering redisplay is not sufficient for it to behave like in
> find-dired, I think.  It needs to handle input and everything as if it
> was running in a different execution thread, like find-dired does with
> the "mapping function" running in an external process (find program).
> 
> > Keep in mind that when Emacs invokes an async subprocess, the display
> > of what that subprocess produces is still done in the same single
> > thread where the Lisp machine runs.  So we already have the
> > capabilities of "pretending" we have multithreading, when these
> > aspects are involved.
> 
> Yes, it works well when the mapping function (in the case of find-dired,
> the find process) runs in a different execution thread/process.
> 
> It does not work well, when it runs in the same single execution thread.

Like I said: output from the sub-process is processed and displayed by
the same single-thread, without any help from additional threads.  the
subprocess produces output, but it is only displayed by Emacs when we
read that output and process it, which is done by the same single
thread which you say cannot do the job.  So the same can be done
without sub-processes.  If needed, of course.

> See find-lisp-find-dired.

Not relevant: that function was not designed provide output during the
search, so it doesn't.

> >> > There are usually protocols for the callback to stop the process and
> >> > prevent the rest of objects from being examined/reported.
> >> 
> >> There is no such thing in directory-files.
> >
> > I think we have something like that in lisp/url/.
> 
> Thanks for the hint.  Unfortunatelly I can't find anything from after a
> quick search.

Grep for "callback".

> > I don't see why this would be non-trivial.  In the simplest case, the
> > callback returns a special value which tells to stop the iteration.
> 
> The simplest case is also the least interesting.

If it does the job, why isn't it interesting?

> > Emacs is not an OS (the popular myth about that notwithstanding).  It
> > provides primitives that are useful for writing editing and
> > text-processing applications.  What you don't find in Emacs is
> > probably outside of its scope, or at least is rarely if ever needed.
> 
> Directory traversal seems to be essential part of Emacs.

And it does work.

> > Where you see something non-trivial, I see just simple use of our
> > unwinding infrastructure.  There's no problem I see with using that
> > infrastructure in recursive processing of directories.  If you see
> > such problems, please tell what they are, specifically, given the
> > unwind_protect functions we have and the mechanism that actually
> > implements the unwinding.  In Emacs, many operations could signal an
> > error or throw, and the user could type C-g at any moment, so this
> > mechanism is in use all the time, and must handle correctly nested
> > unwinders.
> 
> Problem is not with unwind-protect.

I didn't mean unwind-protect (in Lisp), I meant unwind_protect (in C).
We are talking about primitives that should be implemented in C, yes?

> Problem is with directory-files and all functions that use it.  I think
> I pointed out the problems sufficiently.

And I think I explained why I think your assessment is mistaken.

> One fix could be implementing an alternative to directory-files,
> e.g. directory mapping function, plus running it in a extra thread.

You are welcome to try that, but I don't think it will work in Emacs,
not with the Lisp threads we have.

> Another possibility would be to traverse the directory lazily, but that
> brings an issue of eagerly closing directories.

See above: I see no issue.  They could/should be closed in
unwind_protect functions.