Re: Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets)

[Akira Kyle]

On Mon, Nov 23, 2020 at 11:27 PM, Lars Ingebrigtsen 
<larsi@gnus.org> wrote:

> > Thus I feel like the only sane way to mix the two is to allow 
> > gtk to
> > just take over an Emacs window which is always rectangular. No 
> > need to
> > worry about scrolling and all the redisplay optimizations that 
> > will
> > try to interfere with gtk attempting to draw at some point in 
> > the
> > buffer.
>
> I think this design reduces the use cases for emacs-webkit 
> considerably
> -- if all you can do with it is to let it take over a window 
> completely,
> then that's disappointing.  

Maybe it's worth expanding on this since I was originally trying 
to come up a solution that would work more along the lines of what 
you're asking, but ended up where I am entirely due to technical 
reasons and perhaps that journey may be informative...

Consider that any "rich" display element you might want displayed 
among others in a buffer needs to have some display property so 
that redisplay can be aware of it's position relative to other 
elements in the buffer. Thus there ultimately needs to be some 
entry in the glyph matrix for this "rich" display element. But the 
current way redisplay happens, with its various optimizations, you 
can't expect that every time the element is moved or partially 
clipped or even removed entirely, that Emacs will ask for your 
"rich" element to draw itself.

So you have two options. One is you make your "rich" element 
behave pretty much the way images behave and just give Emacs a 
pixbuf to display. But even then you may need nontrivial redisplay 
changes due to Emacs not expecting the pixbuf data to change out 
from underneath itself. The other is you try to do something like 
what xwidget does and synchronize changes in where redisplay 
calculates the position of the "rich" element to go with drawing 
that element yourself (or letting gtk draw it).

I see this second option as ultimately being pretty brittle as it 
needs to force redisplay to draw your glyph every time redisplay 
changes some properties of its display and pass what those are to 
your "rich" element and hope that it respects them. If it doesn't 
or something in this process breaks, your element ends up floating 
in places it shouldn't be because it keeps drawing itself to the 
screen there. I think the former approach of expanding the image.c 
code or creating something modeled after it would be the better 
approach, likely with less visual bugs. Perhaps cairo could be a 
key component of this if its okay to have it as a dependency on 
all platforms. Anyways that was at least the direction my hacking 
on xwidgets was taking me before I got "sidetracked" by this 
dynamic module idea.