bug#15045: Point jumps inappropriately around time of Semantic lexing

[Barry OReilly]

> The easiest solution is to add an optional argument `check-timers'
> to input-pending-p, so you can preserve the exact same behavior.

I assume you mean no-check-timers, nil being default.

> In the long run, we should try and clean up sit-for, but you may not
> have the courage to dig into this right now.

I'm arguing I don't need to dig into sit-for's complexity to maintain
its current behavior. If (sit-for 0 ...) calls something like
thread-yield which in turn calls timer_check, then all ways of calling
sit-for would have unchanged behavior.

Even so, because of other calls to input-pending-p not through
sit-for, there could conceivably be regressions in the latencies of
timers, due to background tasks not yielding when they used to. If we
discovered such cases, the solution is straight forward: replace
(input-pending-p) with (sit-for 0 t) in the offending background task.
But I understand the objection.

If the no-check-timers arg solution is what you want, I'll prepare a
patch to add it and have Semantic and NXML use it.

Also, what do you think are the merits of putting in the
specpdl-walking check discussed earlier, enabled with
--enable-checking? I can't see a legit case where check_timers is
called when point is on an excursion. Such code should not make any
assumptions about the other timers the user is running which might
call redisplay.

> The current state of the concurrency branch is not always
> representative of what it should ideally look like ;-) But even if
> timers are made to belong to a thread, thread-yield should probably
> check timers.

The timer behavior on the concurrency branch seems reasonable. Moving
them to their own threads might be complex, not least because of
concerns that the global lock will be yielded often enough to not
regress timer latencies.