emacs-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Introducing emacs-webkit and more thoughts on Emacs rendering (was R


From: Tomas Hlavaty
Subject: Re: Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets)
Date: Thu, 03 Dec 2020 22:02:47 +0100

On Thu 03 Dec 2020 at 16:39, Eli Zaretskii <eliz@gnu.org> wrote:
>> At the moment, in emacs-framebuffer, I just call it to display an image
>> using call-process-region.  It could be optimized with pipe, but this
>> has low priority for emacs-framebuffer as it is fast enough for my
>> needs.
>
> IMNSHO, neither call-process-region nor pipes are fast enough to be an
> integral part of the Emacs redisplay: they are too slow.  Especially
> if you want to be able to display more than one image at the same
> time.  To see how the performance is inadequate, turn on flyspell-mode
> and scroll through a large text buffer, preferably with lots of
> mis-spelled words (or some text full of special terminology the
> general-purpose dictionary won't know about).  The effect in your case
> will be very similar.
>
>> I am not exatly sure, what the concern is here.
>> 
>> The images do not travel via pipe.  Only meta-data like filenames,
>> coordinates and handles.
>
> Yes, I understood that much.  But that doesn't matter: communicating
> with a spell-checker in the flyspell-mode case also sends just a small
> number of words in each direction, and yet the slow-down is
> considerable, sometimes downright annoying.

Is it an appropriate comparison?

I am sure it is possible to find many examples where Emacs slows down to
crawl even without external process, e.g. a buffer with long lines.
Does it mean I should dismiss Emacs for editing text?

I do not think my use-case is comparable to "scrolling through a large
text buffer, preferably with lots of mis-spelled words (or some text
full of special terminology the general-purpose dictionary won't know
about)".

I do not intent to scroll through a large buffer with lots images.  Even
if I did, there is no evidence that it would suffer the problems in the
above example.  w3m manages just fine.  My use case is something like
image-mode or doc-view or pdf-tools.  emacs-framebuffer already does
that for many image and document types and I have no problems with
speed.

In any case, ability to display images and documents on the console
beats no ability at all.

>> What exactly needs to be done in the middle of redisplay that is
>> relevant to drawing images?  Determining image width and height?
>
> Sending the image spec and receiving the dimensions and other image
> attributes, yes.  And I presume that the sub-process needs some time
> to calculate the answer (because at least some queries require to load
> and process an image).

This is a non-issue.  Images usually do not change those attributes
quickly, if at all.  So this can usually be cached or even calculated
just once.  This is what I do in emacs-framebuffer.

>> > You are supposed to submit data in one of the known formats.  Why is
>> > that a problem?
>> 
>> Because they do not work on the console.
>> 
>> And the reason they do not work on the console is because there is
>> hardcoded dependency on image drawing libraries which do not work on the
>> console.
>
> No, that's not the main reason.  See below.
>
>> > On the console you currently cannot have images in Emacs, period.
>> 
>> Exactly.  And I want this to change.  There is no reason for such
>> restriction.
>> 
>> > This is not because of image libraries, this is because the text-mode
>> > display simply cannot currently support such display elements.
>> 
>> This is not an issue at all.  Capabilities of text-mode display are
>> irrelevant.  w3mimgdisplay is using framebuffer.  I do not know how to
>> better communicate that.
>
> I was talking about the Emacs text-mode display code, sorry for being
> unclear.  The Emacs display code which handles the console is not
> capable of supporting images, or any display element whose dimensions
> are not the same as character cells on a console.  The assumption that
> each display element takes exactly one character cell on display is
> implicitly present in many places in the Emacs display code that
> handles TTYs.  We cannot even display a larger font on a console.
> (There is a special handling of wide characters that take two columns
> on display -- but still the same height as other characters -- but
> that's all.)
>
> To be able to display images on a console within an Emacs text-mode
> frame, you need to be able to tell the Emacs display code that a
> certain area on display is "taken".  But since images will generally
> be larger than a character, in both dimensions, you need the Emacs
> text-mode display code to support display elements of arbitrary
> dimensions, and that is currently not possible.  Adding that would
> need serious changes in the text-mode parts of the Emacs display
> engine code, and that is almost entirely in C.
>
> This is the main obstacle we'd need to overcome if Emacs will ever be
> able to display images on TTYs.  All the rest is much easier.

I understand that.

I do not aim to change that.

Again, my use-case is like that of image-mode, doc-view and pdf-tools.
There is a file, say /tmp/a.png, I open the file in a buffer and the
buffer will show the png image.  Or there is a file, say /tmp/a.pdf, I
open the file in a buffer and the buffer will show the pdf document.

This use-case does not need to deal with the issues outlined above.  It
just needs to know size and position of the buffer, width and height of
characters and screen etc.  emacs-framebuffer already does that and it
is not that difficult, thanks to the hints I already got from you:-)

>> > (You never said you were talking about a console.)  We currently don't
>> > expose console cursor control to Lisp.  Should be easy to add that, if
>> > needed.
>> 
>> I see.
>> 
>> cursor-type does not say it does not work on the console.
>> 
>> I think not all values are doable and relevant.  t and nil should be
>> enough.
>> 
>> Shell I open a bug report for this?
>
> A feature request, yes.

ok, thank you.

>> > I don't think we have such a hook, no.  But if we had it, using it
>> > like you want would cause annoying flickering, because you'd
>> > constantly redraw the image at very high frequency.
>> 
>> I do not think it would cause flickering on the console.  The only thing
>> that normally "flickers" is the blinking cursor.  Emacs does not draw
>> anything if nothing changed.
>
> Emacs doesn't draw anything because it knows what's on the glass and
> what _should_ be on the glass, and compares the two to decide what
> should be redrawn.  But you want to draw on parts of the screen behind
> Emacs's back, so it won't know whether something changed.  And you
> OTOH have no way of knowing where Emacs have redrawn something during
> the last redisplay cycle.  So you will have to redraw your images each
> time Emacs finishes a redisplay cycle, regardless of whether it
> changed anything on the glass in the area of an image.  These
> redisplay cycles happen at very high frequency -- usually, Emacs
> enters redisplay, quickly decides what needs to be redrawn, and then
> does whatever has to be done and exits redisplay.  It tries very hard
> not to redraw the parts that haven't changed, and that is why there's
> no flickering.  But I don't see how you could prevent the flickering
> of those add-on images, because without knowing which parts of the
> screen changed, you will have to redraw them very frequently.

This is not an issue.

You can try and see it easily.

Start emacs on the console.

Start shell in Emacs: M-x shell

Execute: $ cat /dev/urandom >/dev/fb0

Observe what Emacs draws.  I can see blinking cursor.  Part of the
*shell* buffer was drawn and line and column indicators on the
mode-line.  Everything else remains unchanged.

Now ssh from a different computer and run $ cat /dev/urandom >/dev/fb0

Observe what Emacs draws.  I can see blinking cursor.  Everything else
on the screen remains unchanged.  After some time, clock on the
mode-line is drawn.

So if Emacs does not change anything, it does not draw anything on the
console and I do not need to redraw the image to the framebuffer.

The only time I need to redraw the image to the framebuffer is when
emacs changed and drawn something.  But this is a single event, not a
periodic thing.  Because there is no periodic redraw, there is no
flickering.  Unfortunately, there is no hook to run after this single
event.

>> >> Is there such hook?  If not, would it be difficult to add it?
>> >
>> > It won't be hard to add it, but I'm not sure we want to add such a
>> > hook, see above.
>> 
>> That is a shame.  If there was a hook to detect when Emacs has drawn
>> something on the console, I could plug image redrawing there.
>> 
>> If I wanted to add it, where in the code should I look?
>
> In update_frame and its subroutines.  But my recommendation is not use
> such a kludgey approach.

Thanks, I'll have a look.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]