bug#44664: 28.0.50; troubles with some chars in term

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2020-11-19 17:28]:
> > Date: Wed, 18 Nov 2020 23:06:30 +0300
> > From: Jean Louis <bugs@gnu.support>
> > Cc: larsi@gnus.org, schwab@linux-m68k.org, 44664@debbugs.gnu.org
> > 
> > I have been searching to find references:
> > 
> > https://github.com/jquast/wcwidth
> > 
> > https://github.com/streamlink/streamlink/pull/2032
> 
> This is not relevant, Emacs has this data as well.  But since you are
> running a GUI session, the width of characters is not what's
> important; what's important is the width of the font glyphs that Emacs
> uses to display the non-ASCII characters in the term buffer.
> 
> > But I can also see many problems without any wide characters. 
> > 
> > I am also observing various switches of fonts. I have tried setting
> > Terminus font and then I see that when I run mutt that the font
> > changes to something else. After $ reset, it seem to have half
> > Terminus and prompts to be DejaVu Sans, then after several killing of
> > terminal buffer and restarts it started appearing everything to be
> > using Terminus font.
> 
> First, start by trying this in "emacs -Q", to make sure it isn't due
> to some customizations of yours.  Then, to see what fonts are used for
> the "unusual" characters, use
> 
>   M-: (font-at POS) RET
> 
> where POS is the buffer position of the offending character.
> Alternatively, go to the character and type "C-u C-x =", it will pop
> up a buffer with a lot of information including the font.

The default font probably chosen by Emacs at the same point where
there was that one char I think it is making problems is this one:

#<font-object "-GNU -FreeMono-normal-normal-normal-*-15-*-*-*-m-0-iso10646-1">

Video on what is happening with `emacs -Q' is here:
https://gnu.support/images/tmp/2020-11-19-17:43:06.ogv

Otherwise you may see 2 attached screenshots. First screenshot will
show condition "before" as that is when I yet did not come to
allegedly offending characters. There is one line for message of
Thien-Thi

What one DOES NOT SEE is that there is actually another invisible
line, now shown there. I can see it in mutt in xterm, you can find it
under "Web spy" and before "Thien-Thi". That line is not shown in mutt
under M-x term, until I come with the mutt highlighted line to it.

The line is from 2002-03-09 - as I said it cannot be seen.

Then that line shows itself and I can see some replaced characters and
I can see this character not replaced with that circled ?. It is just
perception that this character and maybe others are not properly
interpreted by the terminal or fonts.

             position: 2833 of 7081 (40%), column: 52
            character:  (displayed as ) (codepoint 60531, #o166163, #xec73)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0xEC73
               syntax: w        which means: word
             category: L:Left-to-right (strong)
             to input: type "C-x 8 RET ec73"
          buffer code: #xEE #xB1 #xB3
            file code: #xEE #xB1 #xB3 (encoded by coding system utf-8-unix)
              display: no font available

Character code properties: customize what to show
  general-category: Co (Other, Private Use)
  decomposition: (60531) ('')

There are text properties here:
  font-lock-face       (:foreground "white" :background "black" :inverse-video 
t)
  fontified            t

> Again, what are the two fonts used in this screenshot?  Use the
> above-mentioned techniques to tell, and use "emacs -Q" to eliminate
> the possibility that it's due to your customizations.

Now I have eliminated everything that could point to my
customizations, thank you.

I have also run mutt within M-x term under `emacs -nw -Q' and I find
the same problem is there. At first I cannot even see the line with
2002-03-09. When I then go with the highlighted line from 2002-03-07
message beginning with Spy to 2002-03-10 visible message, the visible
message 2002-03-10 changes itself to 2002-03-09 or inserts the
2002-03-09 message, only then comes to some distortions of visibility
and me, I lose confidence if I am on the right message or not on the
right message.

You may find those screenshots attached showing you condition before I
move with the highlighted line over the invisible 2002-03-09 message,
which is shown later.

> Also, do you have the LS_COLORS environment variable set, and if so,
> what is its value?

Before anything I have removed all colors both from ls and mutt and
removed all control sequences from .bashrc and I feel better without
colors (surprisingly).

Side notes:

- mutt handles these messages in xterm without any distortions

- no other terminal I have like mate-terminal, lxterminal, etc. have
  any visibility distortion on that character

- character referred here may not be the one causing problems and
  multiple characters could be causing problems.

- I was using this file with mutt:
  https://lists.gnu.org/archive/mbox/gnu-emacs-sources/2002-03

1-condition-before.png
Description: PNG image

2-condition-after.png
Description: PNG image

3-xterm.png
Description: PNG image

4-emacs-nw-Q-before.png
Description: PNG image

5-emacs-nw-Q-after.png
Description: PNG image