|Subject:||Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets)|
|Date:||Sat, 21 Nov 2020 20:35:52 -0700|
|User-agent:||mu4e 1.4.13; emacs 28.0.50|
Awhile ago I started a thread on rethinking the design of xwidgets, there was a lot of good discussion about the future of Emacs' rendering capabilities. It personally motivated me to try to better understand how Emacs' redisplay works and just overall better understand the C core of Emacs. As part of that exercise, I decided to see if I could port the existing xwidgets code into a dynamic module. The only xwidget Emacs currently implements is a webkit widget, so this effort turned into creating a dynamic module for embedding webkit in Emacs. It also happens that I've been wishing that I could never leave Emacs and a general web browser was the last daily app that I begrudgingly used outside of Emacs. Anyways here's the result: https://github.com/akirakyle/emacs-webkit.git (sorry for the github hosting, I just have yet to make the jump to another platform -- I guess what they say about vendor lock-in is true).
I was able to implement all of the current features of the webkit xwidget plus some more. I think having this as a dynamic module makes it feel less hacked into Emacs (although still a bit hackey in some places). One interesting additional feature I was able to implement was support for TUI emacs. Essentially it opens a dedicated window for displaying webkit which can be controlled through the keybindings in the emacs buffer corresponding to that webkit view (of course one needs to have running a running graphical session pointed to in $DISPLAY). Perhaps when this code stabilizes, it might make sense to remove the experimental xwidget support from Emacs if this is deemed a worthy successor to such features.
I also wanted to make a few more general comments prompted by the previous thread about the future of Emacs redisplay and rendering given what I've learned in this exercise. I now think its fairly foolish to try to integrate too much of a toolkit library's widgets into Emacs. There will always be this friction between Emacs and the toolkit library wanting to handle the business of drawing to the glass. In a lot of ways I've come away from this very impressed with how Emacs manages to have native toolkit ports to gtk, ns, win, and TUI rendering given the differences in how each wants to handle drawing to the screen, but on the other hand there's a lot of complexity and code behind making this happen, which presents a large barrier to entry to people like me who may want to hack on the lower level drawing capabilities of Emacs. We've also seen hardware change a lot since Emacs display machinery was written. CPUs have gotten faster and now everyone has a GPU. I think there's a potential for Emacs to become both snappier and more extensible with respect to its rendering. I had initially argued that Emacs probably shouldn't deal with rendering on the level of opengl, however I think as I've understood better the way Emacs currently handles rendering everything, that's exactly the direction Emacs should move in, *if it can*.
So here is my new pie-in-the-sky proposal for what the future of Emacs rendering could look like. Given the success and hopefully soon-to-be in master native-comp branch, perhaps it may not be so crazy to think about moving more rendering to elisp. If a clean elisp interface to opengl were defined and Emacs were able to render itself using this optimized elisp opengl interface, the low level code for each platform could be minimized to just providing the opengl surface and exposing input events. In fact there may be ways in which elisp is well suited to the way modern GPU graphics are rendered using scene graphs. In fact gtk4 is moving its rendering to such an approach in order to take advantage of hardware graphics acceleration. I think a first step would likely be to focus on implementing an elisp interface to cairo as Emacs is already using more of cairo to render itself. If it seems like the performance is good and elisp can represent graphical operations well, than that might be a good indication that such a model could be successful path forward towards an Emacs that could bring its extensibility to its own basic rendering.
|[Prev in Thread]||Current Thread||[Next in Thread]|