Re: [PATCH v3] Inline image display as part of a new org-link-preview system

[Karthik Chikmagalur]

>> I'm not sure how to solve this problem.  Here's what I'm picturing --
>> note that this doesn't actually avoid the process sentinel pile-up:
>>
>> The preview provider should return its callback function to Org so Org
>> can run it at its convenience.
>>
>> Org runs (funcall #'preview-func ov path link)
>>
>> preview-func returns either
>> 1. t, if preview is synchronous and successful,
>> 2. nil, if preview is synchronous but failed,
>> 3. A callback-func, if preview is asynchronous.  In this case the
>> preview-func starts the network request or async job.
>>
>> Org adds callback-func to the queue of callback-funcs for this run, and
>> runs them spaced out on a timer or something.  Preview success depends
>> on whether the callback-func returns t or nil.
>>
>> Implementing something like this should be easy with org-async, included
>> in the LaTeX preview patchset.
>>
>> But the process sentinels will run independent of Org's callback queue.
>> The callbacks only place the resulting image (or other metadata) on the
>> overlays.  The bottleneck here is not the display engine, so this
>> doesn't solve the problem.
>>
>> Can you give me some more details of what you have in mind?
>
> I mostly meant calling preview-func asynchronously, while idle, spaced
> out, spending not longer than a fraction of second to call several
> preview-funcs.
> Spacing might then be controlled by the users.
>
> We might go further, and let the preview functions return a
> process. Then, we may explicitly control running sentinels just for that
> process via `accept-process-output'. But I am not sure if we need to go
> that far.

Do you mean something like this?

(while (re-search-forward org-link-any-re end t)
  ;; Make overlay ov here
  ;; Find path, link and preview-func here

  (push (list ov preview-func path link) previews-remaining))

(dolist (preview-data-chunk (seq-partition previews-remaining 6))
  (run-with-idle-timer
   0.10
   (lambda (preview-data)
     (pcase-dolist (`(,ov ,preview-func ,path ,link) preview-data)
       (when-let ((buf (overlay-buffer ov)))
         (with-current-buffer buf
           (if (funcall preview-func ov path link)
               (push ov org-link-preview-overlays)
             ;; Preview was unsuccessful, delete overlay
             (delete-overlay ov))))))
   preview-data-chunk))

Where the chunk size (6) and the idle time (0.10 seconds) will be
customizable.

Karthik