bug#45117: 28.0.50; process-send-string mysteriously exiting non-locally when called from timer

[João Távora]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> No, I was no presuming such a simple model, actually.  I was really
>>> thinking about "send data to the LSP server then get some answer
>>> a second or more later".
>> Right, so in LSP it's perfectly possible to send three requests in a
>> row, say reqX, reqY and reqZ and get three replies in a completely
>> different order repZ, repX, repY.  How to you match each reply to each
>> request?
>
> I assume there's some "request-id" mechanism.  Not sure what this has to
> do with this discussion, OTOH.

Right, a request-id mechanism.  The request must be registered
atomically with process_send_string.  If you interrupt in between, you
have inconsistent requests (either registered request-id's for which no
actual request was fired, or requests which were fired for which no
request-id's were registered).  Client code can detect/prevent these
interruptions, but it's clumsy.  And may cost the dev many hours to
understand what is up.  Shouldn't be default IMO.

>> process_send_string may send things in "bunches", I read in the
>> docstring, but it will not (and should not) be interrupted.
>
> Indeed, I believe it should not be aborted in the middle by
> `while-no-input` (it would be a bug, because the `process-send-string`
> API doesn't offer any way to know what has been or hasn't been sent in
> that case).

Agree.

>> But that is not always so.  And I think it's too eager of ElDoc to try
>> to do that so early and so brutally.  It's better to leave it to the
>> callback handlers, which we have now.  That's a much safer spot to
>> know if the answer we just got still makes sense.  Or if we're in
>> a hurry, we let the backend know asap.
>
> You might be right: the result of the current request sent to the LSP
> could still be useful for the next eldoc-idle-time cycle, indeed.

Yes, it's only an heuristic.

>>> The contract is different for timer functions than it is for eldoc
>>> functions, yes.  This is because the expectation is that eldoc functions
>>> may run for a non-negligible amount of time.
>> Why do you have that expectation?  Any particular example in the wild?
> Good question.

:-)

>> It was, after all, the status quo after you changed it for 27.1.
>> Perhaps you had a rationale?
>
> I probably did, but ... can't remember and wasn't clever enough to write
> it in the commit message :-(
> Maybe to accommodate those backends which needed async operation but had
> to live within the confines of the previously limited eldoc API?

Likely, yes.  But which one of those were the "blocking" type?  Because
even with the limited API, SLY/SLIME were just calling eldoc-message
from the process filter. Which is equivalent to what we have now in
terms of sync.  In fact, to keep backward compatibility, I haven't
touched this part of SLY at all.  Anyway, you must have done it for some
other slow synchronous, wait-for-the-retval backend.  That hypothetical
backend will hurt if we take back the while-no-input.  OTOH that
hypothetical backend can now upgrade to a much better API.

> Maybe the maintainer of eldoc.el will prefer to undo this change,
> then ;-) ?

Who's that? ;-) But OK, eldoc.el is now distributable independently so
we have good defense against this.

João