bug#39286: 27.0.60; Wrong font for Cyrillic characters in emacs buffer

[Eli Zaretskii]

> Date: Fri, 31 Jan 2020 12:13:18 +0300
> From: Дмитрий <jumper047@gmail.com>
> Cc: 39286@debbugs.gnu.org

A year later, I bumped into a system where I saw something very
similar, so I debugged this.

> ((open "-outline-BatangChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "harfbuzz:-outline-BatangChe-normal-normal-normal-mono-13-*-c-*-gb2312.1980-0")
>  (sort-by
> "-*-normal-normal-normal-*-13-*"
> "harfbuzz:-outline-BatangChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0") 
> (list
> "-outline-*-mono-*-gb2312.1980-0" 
> ["-outline-BatangChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GungsuhChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GulimChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-DotumChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-MingLiU-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Gothic-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Mincho-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-NSimSun-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-FangSong-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-SimHei-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-KaiTi-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-BatangChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GungsuhChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GulimChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-DotumChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-MingLiU-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Gothic-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Mincho-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-NSimSun-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-FangSong-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-SimHei-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-KaiTi-normal-normal-normal-mono-*-c-*-gb2312.1980-0"]) (w32font-list
> "-outline-*-mono-*-gb2312.1980-0" 
> ("-outline-BatangChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GungsuhChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GulimChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-DotumChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-MingLiU-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Gothic-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Mincho-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-NSimSun-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-FangSong-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-SimHei-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-KaiTi-normal-normal-normal-mono-*-c-*-gb2312.1980-0")) 
> (harfbuzz-list
> "-outline-*-mono-*-gb2312.1980-0" 
> ("-outline-BatangChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GungsuhChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-GulimChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-DotumChe-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-MingLiU-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Gothic-normal-normal-normal-mono-*-c-*-gb2312.1980-0" "-outline-MS
> Mincho-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-NSimSun-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-FangSong-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-SimHei-normal-normal-normal-mono-*-c-*-gb2312.1980-0"
> "-outline-KaiTi-normal-normal-normal-mono-*-c-*-gb2312.1980-0")) (list 
> "-*-Courier New-*-gb2312.1980-0"
> nil) (w32font-list "-*-Courier New-*-gb2312.1980-0" nil) (harfbuzz-list 
> "-*-Courier New-*-gb2312.1980-0" nil)
> (list "-*-Courier New-*-mono-*-gb2312.1980-0" nil) (w32font-list "-*-Courier 
> New-*-mono-*-gb2312.1980-0"
> nil) (harfbuzz-list "-*-Courier New-*-mono-*-gb2312.1980-0" nil) (list 
> "-outline-Courier New-*-gb2312.1980-0"
> nil) (w32font-list "-outline-Courier New-*-gb2312.1980-0" nil) (harfbuzz-list 
> "-outline-Courier
> New-*-gb2312.1980-0" nil) (list "-outline-Courier New-*-mono-*-gb2312.1980-0" 
> nil) (w32font-list
> "-outline-Courier New-*-mono-*-gb2312.1980-0" nil) (harfbuzz-list 
> "-outline-Courier
> New-*-mono-*-gb2312.1980-0" nil) (list "-*-iso8859-5" nil) (w32font-list 
> "-*-iso8859-5" nil) (harfbuzz-list
> "-*-iso8859-5" nil) (list "-*-mono-*-iso8859-5" nil) (w32font-list 
> "-*-mono-*-iso8859-5" nil) (harfbuzz-list
> "-*-mono-*-iso8859-5" nil) (list "-outline-*-iso8859-5" nil) (w32font-list 
> "-outline-*-iso8859-5" nil) (harfbuzz-list
> "-outline-*-iso8859-5" nil) (list "-outline-*-mono-*-iso8859-5" nil) 
> (w32font-list "-outline-*-mono-*-iso8859-5"
> nil) (harfbuzz-list "-outline-*-mono-*-iso8859-5" nil) (list "-*-Courier 
> New-*-iso8859-5" nil) (w32font-list
> "-*-Courier New-*-iso8859-5" nil) (harfbuzz-list "-*-Courier New-*-iso8859-5" 
> nil) (list "-*-Courier
> New-*-mono-*-iso8859-5" nil) (w32font-list "-*-Courier 
> New-*-mono-*-iso8859-5" nil) (harfbuzz-list
> "-*-Courier New-*-mono-*-iso8859-5" nil) (list "-outline-Courier 
> New-*-iso8859-5" nil) (w32font-list
> "-outline-Courier New-*-iso8859-5" nil) (harfbuzz-list "-outline-Courier 
> New-*-iso8859-5" nil) (list
> "-outline-Courier New-*-mono-*-iso8859-5" nil) (w32font-list 
> "-outline-Courier New-*-mono-*-iso8859-5" nil)
> (harfbuzz-list "-outline-Courier New-*-mono-*-iso8859-5" nil) (default\ 
> fontset:\ font\ for 1092 nil))

Note that iso10646-1 charset never appears in the above log, although
the setup for Cyrillic in the default fontset is this:

     (cyrillic ,(font-spec :registry "iso10646-1" :script 'cyrillic)
               (nil . "ISO8859-5")
               (nil . "microsoft-cp1251")
               (nil . "koi8-r"))

IOW, iso10646-1 is the first registry value Emacs should have tried,
but it is nowhere to be seen in the above log.

The reason for this is that the fontset is reordered according to the
locale's preferences, as recorded in Vcharset_ordered_list and
Vcharset_non_preferred_head, and these are screwed up because Windows
returns to us a bogus locale name "ZZZ".  That causes the iso10646-1
part of the Cyrillic fontset to be pushed way down the preference
order, and we begin by examining the other registry values.

Even so, iso8859-5 should have saved the day, but there's another
problem: we don't have information in w32-charset-info-alist for
mapping iso-8859-5 into Windows charset constants.

So I have now fixed both problems on the emacs-27 branch, and as
result the Cyrillic characters are displayed using Courier New, as
expected.

Do you still have access to that system where you saw the problem?  If
so, can you build Emacs from the recent emacs-27 branch and see if the
problem is solved?  Even if you cannot build Emacs, I can tell you how
to fix the problem manually.

Thanks.