bug#54970: 28.1; Some emoji no longer display

[Howard Melman]

> On Apr 23, 2022, at 3:51 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> First, is composition supposed to work everywhere?
> 
> Yes -- as long as the font used for the characters supports that. 

> Please discuss this with the developer of which-key

Ok, I've opened a bug with which-key to do that.  Thanks.
I had also found the which-key code involved but didn't 
find any references to composition in it or other obvious
things it did to disable it and wanted to confirm that it 
probably shouldn't have to do anything to enable it.

>> Second, I'm using this now for emoji fonts:
>> 
>> (set-fontset-font t 'symbol '("Apple Color Emoji" . "iso10646-1") nil 
>> 'prepend)
>> 
>> I saw from a related reddit post that the font Symbola has some other glyphs
>> that seemed of interest.  It renders things as plain graphics not as color
>> emoji which is fine (or preferred) for math symbols.  I'd like to use Apple
>> Color Emoji first and then if it doesn't support a character fall back on
>> Symbola.  I thought to do that with:
>> 
>> (set-fontset-font t 'symbol '("Symbola" . "iso10646-1") nil 'prepend)
>> (set-fontset-font t 'symbol '("Apple Color Emoji" . "iso10646-1") nil 
>> 'prepend)
>> 
>> But it didn't work.  Most emoji were rending as Symbola.  I then removed the
>> first line and restarted emacs (so just my original line is in the init) and 
>> emoji
>> were *still* rendered with Symbola.  It wasn't until I disabled the font in 
>> the
>> mac utility Font Book.app that it went back to full color emoji rendering. I 
>> recall
>> a similar issue a year ago trying to get Julia Mono working too.
> 
> What was the value of use-default-font-for-symbols?  If it was
> non-nil, Emacs will in general try to ignore the fontset when it needs
> a font for a character belonging to the 'symbol' pseudo-script.

It is t, which I believe is the default value.  The manuals don't mention
use-default-font-for-symbols so I didn't know about that. 

> Also, what does the fontset entry for the relevant character(s) say?
> (Invoke "M-x describe-fontset RET fontset-default RET" and look for
> the entry that corresponds to the codepoints of the characters you are
> interested in.)  Note that the default fontset with which Emacs starts
> already includes references to Symbola, even without any
> customizations.
> 
> And what else is in your init files, besides those two lines?

I can reproduce this in emacs -Q so the answer is nothing.

Sorry I didn't mention a specific character, but it was
a lot of them and it's hard to lookup codepoints
for each of them.  Here are ones I saw from a file:

🥇 🥈 🥉 💰 ✅ 👍️ 👎️ 👋️ 🤦‍♂️️ 🤷‍♂️ ⭐️ 📺️ 🎥 ♠️ ♣️ ♥️ ♦️
😡 😢 🙄 😂 🤣 😀 🤔 😉
1️⃣ 2️⃣ 3️⃣ 4️⃣ 5️⃣ 6️⃣ 7️⃣ 8️⃣ 9️⃣ 0️⃣
🔨 🎯 🗡 ❄️ 🔥 ⚡️ ☠️ 💥 🙏🏻 👉 🔭️ 💪 ⏳ 😖 ⏩️ 👬 ⏪️ ♻️

And there were may others I saw doing insert-char with
a completion system that showed and rendered the characters.

So I worked with 0x1F528 (HAMMER) a level 1 emoji who's
default rendering is emoji, not text.  Checking describe-fontset
for it I see:

📿 .. 🔽 (#x1F4FF .. #x1F53D)
    -*-Noto Color Emoji-*-*-*-*-*-*-*-*-*-*-iso10646-1
    -*-Symbola-*-*-*-*-*-*-*-*-*-*-iso10646-1
    -*-*-*-*-*-*-*-*-*-*-*-*-iso10646-1

I didn't realize emacs knew about Symbola (why would I?).
And I had forgotten that the scripts symbol and emoji do
not overlap at all (because from the previous issue setting
the emoji script to use Apple Color Emoji (ACE) didn't cover many
characters but setting it for symbol did cover them and seemingly
every emoji.

It turns out HAMMER is in script emoji, so setting the fontset
for 'symbol didn't tell emacs to use ACE for it.  With Symbola
enabled emacs found it and used it.  Without Symbola enabled
emacs didn't find it and I guess searched for a font that includes
this character and found ACE.

So now I'm doing this:

(set-fontset-font t 'symbol '("Apple Color Emoji" . "iso10646-1") nil 'prepend)
(set-fontset-font t 'emoji '("Apple Color Emoji" . "iso10646-1") nil 'prepend)

and it seems to work.  I see emojis I've previously used rendered
in ACE and I see some characters that ACE doesn't cover, like
0x 2b59 (HEAVY CIRCLED SALTIRE) rendered in Symbola.
With Symbola enabled I now need to use both because emacs
can find a font other than ACE that can render script emoji.

I know the definition of the set of emojis is difficult but also that it
doesn't overlap with the intuition of a user not steeped in Unicode. 
Perhaps this is particularly bad on a mac where ACE is used 
universally to render emoji and the system emoji picker (shared
with iOS) includes as emoji many characters in both the emoji
and symbol scripts.  And I understand that you don't want to include
references to ACE in emacs.

I think I would have benefited from a describe-scripts command
that some how showed what ranges these scripts cover in a way
easier than using C-u C-x = for each character.  Or maybe a
function scripts-used-in-region that returned a list of the scripts
of each character.

I do think I and other mac users would benefit from a previously
mentioned possibility of a setting handling those emoji who's default 
rendering is text to either render as emoji or include them in
the emoji script (instead of symbol).

I'm not really clear on categories in emacs.  Maybe an emoji
category that included characters from both script emoji and
some from script symbol and a way to use it to modify a
fontset?  The Categories section of the elisp manual doesn't
say where emacs uses them but I see them used in properties
and regexps and an easy way to regexp search for any character
in an emoji category would be nice.

Back to the original post in this bug, the NEWS instructions to:
    (set-fontset-font t 'emoji '("My New Emoji Font" . "iso10646-1") nil 
'prepend)
confused me because it didn't seem to have any effect on macos.
If this (or some other) instruction included "and you may want to do the same
for the script symbol though it might include more characters you
desire" it would have helped.  Particularly if it recommended
this for mac users.  I don't see a mention of emoji in the emacs
manual except in the antinews.  Perhaps it could mention that emacs
by default knows to look for Noto Color Emoji or Symbola for
many emoji characters and how systems without these fonts
might be configured effectively.

Howard