bug#41531: 27.0.91; Better handle asynchronous eldoc backends

[João Távora]

On Wed, May 27, 2020 at 10:14 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> > No, the creditor of the future or issuer of the callback aborts or
> > invalidates the previous version just before issuing a new one. Nothing
> > pre-command-hook here.
>
> Where/when would eldoc-mode do it?

In the idle timer function.  That's when it decides it wants
new info, and so any old info that hasn't arrived yet is probably
out of date.

But I think I understnad what you mean pre-command-hook.
You suggested pre-command-hook because that's where you
can check if point is far enough away from the place where
the original request originated?  That could work, but I'd prefer
that the info to _do_ come in, because after that info is about
the  last time my point hung out at a particular position long
enough to trigger a request.  This is how Eglot, SLY and
SLIME work, btw.

> > Invalidation may or may not entail letting the
> > holder of the callback know that the previous call became invalid.
>
> Letting know the issuer of the future, you mean.

Or the holder of the callback depending on which abstraction
we're talking about.  I define the holder of the callback as
he who is about to call it.

> > Flymake does this: by invoking a backend again with a new callback
> > instance it is signalling that the preceding one became invalid.  If the
> > backend tries to call the previous callback, it is an error and the
> > backend is disabled.
>
> Worse is sometimes better, we know.

That is a very confusing statement.

> >> It's good to have a well-documented contract. Functions do
> >> _everything_. They can't be optimal for everything.
> >
> > You're missing a Lisp point here.  It doesn't matter if it's an CLOS
> > object, a struct, a function or my beautiful singing voice: it just has
> > to be an object which you can make unique instances of and can respond
> > to funcall, still-wanted-p, (setf still-wanted-p), errored-p, and (setf
> > errored-p).  That's the contract.  A function fits perfectly.
>
> That would be my "alternative" suggestion: for
> eldoc-documentation-functions elements to return a function (denoted as
> FETCHER in the docstring) if they want the async convention.

They need to _receive_ an object produced externally, somhow.
If they return a function as youv suggest, they are only doing so
they can later _receive_ another object.  This is needlessly
complicated.  To receive objects in some place, just use argument
the argument list of that place.
>
> >>>>> The future's creditor is the only one who could do that to any
> >>>>> useful effect.  Does it have access to the process?  Probably not.
> >>>> It can (barring any complex abstractions). It created the process,
> >>>> after all.
> >>> Not really, it asked a client to solve a problem, doesn't know how
> >>> the client if the client is doing by async process or cointoss.
> >> Seems like we're miscommunicating.
> >
> > Well you implied that the creditor of the future (the caller who
> > received) created the process.  It does not.  See the patch to Stefan.
>
> Okay, creditor != creator. But what you've said a few messages back
> (seen at the top of the quotes chain above) doesn't make sense: the
> creditor will call (future-abort fut), and the issuer of the future can
> make sure that this operation will indeed kill the process.

No, it does make sense. Read it again. What you're saying is what I
meant. But that still means that the process sentinel will have to deal
with out-of-band kills that it must distinguish from other out-of-band
kills (such as, say, a kill -9 from the shell). That is added complexity.

It is better, in my opinion, to make this softer.  Let the creditor
signal, I don't need this anymore, and the issuer will take appropriate
measures synchronously, i.e. in the process filter and not in the
process sentinel.

> That's the main idea behind aborting futures. Or canceling. Whatever
> term we're going to pick.

But, again, nothing you're describing here can't be implemented
with passing a callback in the arglist. It's independent.  Futures
particularly the versions you and Stefan are proposing are just
other places to type basically the same sexps.  They're a stylistic
change over callbacks, but nothing more, fundamentally.

> >> See above about not having to change anything.
> >
> > But then we don't have to change anything in any case!  I already
> > changed EVERY user of eldoc-documentation-functions: every single one of
> > the 5 in existence in the entire world.  So we're all good.
>
> And then we'll need to change them back.

Nothing of the sort.

> And in the meantime, the new
> convention could get external users (some people do live on the bleeding
> edge), and this will get messier.

That's only if you assume you'll break compatibility, because you
dislike callbacks so much that you want to close off the arglist
forever.  But not only may users may not dislike them, they may
even prefer them. And keeping the arglist open, not closed, is a
very good idea for extensibility functional API's anyway.

> >> OK, I see your point: eldoc-documentation-functions is new. And
> >> apparently you don't intend to add this feature to the variable
> >> without "s".
> >
> > Yes, exactly.  eldoc-documentation-function should be obsoleted IMO.
>
> Perhaps. I'm also not buying the usefulness of eldoc-documentation-compose.

Yeah,I don't get it particularly, either.  I mean, I can see its uses.
but I'm glad you're finally getting the overengineered feeling :-)
And it's waay more useful than futures here.

> >>> It just looks like you're holding this problem hostage to introducing
> >>> some kind of rushed futures solution.  I don't agree with either of
> >>> these things.
> >>
> >> Who's holding what hostage? I showed a smoother approach, you didn't
> >> like it. No big surprise about that.
> >
> > Let me explain. First: it's clearly not "smoother", your're asking users
> > to wrap their heads around a function that returns a function taking a
> > function.  That's not what I want to present to Eglot contributors, for
> > instance.
>
> Would they need to? As soon as an existing Eglot's implementation is in
> place, that exact part of the code wouldn't need to be touched often.

Code is read much, much more than is is written.  And WTF per minute
_are_ a measure of code quality.  I would like to avoid this particular
WTF please.

> In any case, you are over-exaggerating. This exact design has been a

"over-exaggerating".  Very meta.

> part of "asynchronous" backend calling convention in Company for years.

People will use what you give them, if you they have no other option.

> And not once have I seen a complaint that it's overly complex.

Anyway, count one now.  Or don't.  I mean, I really dislike
company-backends throughout, but I don't use it, either. I mean
I know you inherited it and I had to hack on it to add flex support
for the capf thing, but it's a strange re-implementation of functions
to have one single function that does wildly different things based
on one argument.   I guess at the time there weren't generic
functions. And the cons :async thing is equally confusing to me.
Sorry to be frank. But I guess some people will love it, for some
equally legitimate reason: it will seem "right" to them, or "clever".

> > And I'm not too crazy with presenting them this "future
> > thing" that is completely different from Eglot's use of Flymake,
> > jsonrpc.el, completion-at-point, etc...
>
> Didn't you say that Flymake could use futures as well?

It could, sure, especially if you have a thousand futuristic devs
itching to write backends but not grokking callbacks.  But let's
not kill the existing API, please.

> > In other words, my ambition is
> > consistency and you seem to be denying it for reasons I can't
> > understand, because nothing in the steps I am taking denies _your_
> > ambitions, which seem to be futures.  That's why I speak of "hostage".
>
> See above. But perhaps we should after all suspend this discussion until
> we had a chance to reach a better mutual understanding of the futures
> API, and how we expect it to help (or not). I promise to show some
> proposals in the near future.

Suspend this discussion?  Sure, this discussion yes, if your want.
But not this bugfix: that would be exactly what "holding hostage"
means. Don't hold this bugfix hostage: it has nothing to do with
futures.

João