I don't know a way to get continuous updated output in an org-buffer
Thanks for the response. I didn't necessarily expect continuous output into the org-buffer itself to work, but I don't see why the Python subprocess can't display output as it occurs. After all, it uses comint, and comint certainly has facilities for collecting output incrementally while still displaying it (cf comint-output-filter-functions).
It looks to me like the problem is that org-babel-comint-with-output uses a "while" loop to collect process output (ob-comint.el:92). At least, it could insert the output into the subprocess buffer and make redisplay happen.
But I'm not sure why the code is written that way anyway. Long running "while" loops in Emacs code don't seem like a good idea to begin with. Wouldn't the more natural way for this code to be written in Emacs be the following?
- an output filter gets added to the subprocess that collects output - the code is sent to the subprocess for execution - the command returns - the output filter inserts any data it gets into the subprocess buffer, into its "results" data structure, perhaps even into the org-buffer - when the output filter gets the eoe-indicator, it removes itself from the output filter list and sends a notification that execution has completed
If the user schedules a second block for execution, the simplest thing to do is return an error if there is already a block executing for that subprocess; alternatively, it could be queued somewhere.