bug#25943: 21.5 Frame Display Difficulties

[martin rudalics]

> I am a bit surprised by the content of your reply.  I wonder if I have
> offended you; if I have I apologize; I assure you I do not know how, nor
> was it intentional.

You have not offended me.  Quite on the contrary, your messages were a
pleasure to read and I have to apologize for the content of my reply.
Please forgive me for its rude tone.

> In view of what you have written I do think that I need to do a reset and
> try to make clear how I see things.  I have core code, extended over
> years, that has run successfully on emacs 18.n, 19.n, 20.n, 21.n, 22.n,
> and 23.n.  The code, and emacs, have run successfully on numerous
> platforms and operating systems, the ones I remember being DEC/Unix,
> IBM/aix, PC/Windows NT, and PC/Linux(various).  I cannot remember much
> about the window managers I have used, I just take what is there.  Over
> the years I have become increasingly grateful for emacs because emacs has
> kept me insulated from the vagaries of the hardware and software on which
> I have been running.  When moving to a new version of emacs, sometimes I
> have needed to make minor changes, and, since I have a small amount of C
> code that is necessary for me, I have always needed to modify a few C
> files, make update patches, etc.; but that is all, the lisp has gone on
> forever.  Upgrading has been simple and, crucially, _possible_!  It is not
> always possible with software to rehost.
>
> I want to emphasize the "insulated" part of what I have written.  I have
> successfully transferred my code from emacs version to version, host
> machine to machine, run across the internet running on one OS and
> displaying on another, etc.; and all the time emacs has, very simply, got
> on with it.  I am constantly staggered by the flexibility of emacs, at any
> time, and over time, and its ability, I repeat, to insulate me from the
> hardware and software changes that are occurring underneath the code that
> I write.
>
> Now I have to upgrade my OS and emacs version again.  This time a couple
> of
> nasty things have happened.  It is very simple: emacs 25.1 has failed.  I
> have no reason to think that my code is faulty; it has not been changed
> and the same lisp code (not even a copy) is running right now on emacs
> 23.2 as it has run for a long, long, time on other versions.  My code runs
> on 25.1 except for a couple of, for me, serious, display bugs.  I
> submitted a bug report to the best of my ability; and I still hope that
> the bugs will be repaired; I still want to use emacs.

Thanks for your explanations.  Sadly Jan Djärv - our only colleague
skilled in developing the Emacs-GTK/X11 interface - has left Emacs more
than a year ago and we are now left with a couple of unresolved bugs and
without any assistance in this area.  Moreover, pressure from recent
releases of GTK increases as well (compare Bug#25851) so I'm afraid that
Emacs will not be able to keep up pace with your expectations.

> The above is my starting point, not what you have written.  In addition,
> there are other things that are disturbing.  One is that you write
>
>      "As a rule, users should not specify frame positions manually.  Window
>       managers dislike it.  Not because they think they are better at it
>       but because it disturbs their usual flow of control."
>
> So, for the first time in a quarter of a century, I, an emacs user, have
> to concern myself with the window manager that I am using?  Also: "While
> this might work some of the time with some window managers I wouldn't rely
> on it."  So, successfully transferring emacs and my code in the way that I
> have in the past has been thrown out of the window, emacs is no longer
> going to handle such things as it has done so magnificently in the past?

The rules I sketched were not aimed at you in particular.  These rules
have been collected from commentaries in the sources of Emacs, GTK, GDK,
X11 and miscellaneous bug reports.  They comprise more or less what we,
in our documentations, have concealed in this regard so far.  I plan to
eventually update the Elisp documentations accordingly.

Most problems in this area result from attempting to position the
initial or a new Emacs frame - a frame whose (typically X) window has
not been created yet.  If such a frame is made visible "soon",
positioning it more likely succeeds but has the unpleasant side-effect
described by N. Jackson as

  As for additional flicker, when I start Emacs things jump about like a
  scalded cat, including a phantom random vertical scroll bar that often
  makes a momentary appearance in the middle of my display. I've never
  really understood why Emacs does this then other programs seem to manage
  without such problems, but I've come to accept the behaviour -- after
  all, it's only momentary while Emacs starts up ...

and summarized by you as

  Starting with a visible frame is tacky because of the flashing that
  results.

It would be nice if we were able to make your approach work in general
and ask Mr. Jackson to use it too but for doing that we would need your
help.

I don't know why your code has worked successfully with Emacs 23 and
doesn't work with Emacs 25 any more.  One way to find out is by
bisecting the Emacs sources (I can't do that here because it would take
me a couple of days with my slow hardware).  And even if we find the
offending commit, we might not be able to revert it because it could
have helped to solve a similar problem with another window manager.
Anyway, if you have git and the necessary resources, bisecting would be
a valuable first step.

> The second disturbing point is "If you really need to position and size a
> new frame, please stick to the following rules:".  Sorry, but no.  I am
> not a cowboy programmer; also I am quite compulsive about sticking to
> conventions and rules; generally, I find that to be good practice.  But
> the rules that you have announced are there for the wrong reason: emacs
> 25.1 cannot do what its predecessors have done.  It should not be the
> responsibility of a user to cover the failings of a new version of emacs.

I fully agree with you.  Here, Emacs 26 has become too slow to do many
things I was able to do with Emacs 23 and if it were not for a few
features I try to develop, I'd certainly stick with Emacs 23 instead.

> Thirdly, do you really think that my code is so simple, or so buggy, that
> I can zip in and change a couple of things to get it working again?  I
> hope not.  Also, if a function such as set-frame-position exists and is
> documented, then it should be usable and it should work.  Otherwise,
> there is a bug.

There _is_ a bug.  Unfortunately, there's not only one bug but quite a
number of them.  And so far I found only a somewhat hackish solution to
fix them here.  I'll eventually push a "fix" to the repository but this
will certainly not become part of the Emacs 25 series.

> This thread has gone off track, regrettably.  Here is my summary of the
> present status of the three problems I have had with 25.1:
>
> Problem one [initial display of a new invisible frame is not correctly
> placed] has been identified clearly.  This problem can be demonstrated
> with simple lisp code.  The problem only occurs when a toolkit is used.
> There are a couple of get-arounds.  The problem source has not been found.

The problem source is that ‘set-frame-position’ calculates an incorrect,
off-screen position for the frame and that your window manager corrects
that position with on-screen settings of its choice.  You should be able
to verify this claim by running Emacs under gdb, putting a breakpoint at
the beginning of the function x_set_offset in xterm.c and retrieving the
values of modified_left and modified_top as passed to XMoveWindow.

> Problem two [initial display of a new visible frame can involve
> non-clearing of the terminal input buffer] has been identified mainly by
> its solution, which is appropriate use of (discard-input).  This problem
> can be demonstrated with simple lisp code when it occurs, which may be
> machine dependent.  The problem source has not been found.
>
> Problem three [buffer display in a new frame can be truncated] has not
> been demonstrated with simple code.  However, this is a serious display
> problem that has arisen with 25.1.  The problem source has not been found.

Both of these problems might be related to the fact that you start with
an initially invisible frame and the redisplay engine doesn't catch up
with the new dimensions when the frame gets eventually exposed/mapped.
There have been some code changes wrt the display engine redrawing a
frame only when it's needed, so maybe we should look there.  But I'm
afraid that it might not make much sense to investigate the sources of
these problems as long as they are potentially shadowed by the first
problem.

> I hope both that the above is a reasonable description of the problems,
> and that progress can be made on these problems; I am willing to help to
> the extent that I can.

Please do that.  And even if you're reluctant doing that, please try to
set up your frame by using the 'left and 'top frame parameters.  At the
very least doing that should confirm that ‘set-frame-position’ is indeed
the major culprit.

Thanks, martin