bug#20727: 24.5; Font fallback doesn't work for the Emoji range

[Eli Zaretskii]

> Date: Sat, 13 Jun 2015 10:07:09 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: v.schneidermann@gmail.com, andrewjmoreton@gmail.com, 
>  20727@debbugs.gnu.org
> 
> > The Symbola font looks much more crisp on my system, FWIW.  Which font
> > back-end did you configure Emacs to use?
> 
> Sorry, I don't know what "font back-end" means.

The answer is here:

>    Does Emacs use -lm17n-flt?                              yes
>    Does Emacs use -lotf?                                   yes
>    Does Emacs use -lxft?                                   yes

This means your font back-end is xft with libotf.  That's the most
advanced one we have on GNU and Unix systems, I think.  I thought
maybe you had something less advanced for some reason.

> > Also, if you use the iso10646-1 variant instead of the iso8859-1 as
> > the default font, doesn't that fix the problem
> 
> Yes. The problem, I imagine, is that users will have put a fixed-width font 
> of 
> their own preference into their .Xdefaults or .Xresources file (e.g., 
> "Emacs.font fixed").  This is reasonably common, and the recent change makes 
> symbols look much worse, at least in my environment.

The question is how to avoid that without re-introducing the original
problems.

> By the way, I can reproduce a similar problem with "emacs -Q -font fixed".  
> It's 
> not as unreadable there, but it's still pretty bad: Symbola characters have a 
> different width than the fixed-width characters that Emacs previously used, 
> so 
> source code listings no longer line up.

At some point, with sufficiently rare characters that are not well
covered by fixed-width fonts, this will always happen.  I don't think
it's a catastrophe; it is certainly better than having the boxes with
hex codes instead (which also make text misalign, btw).

> > existing
> > fonts are frequently inadequate, in that they claim support for
> > Unicode ranges where they actually support only a handful of glyphs.
> > Users then complain that they have decent fonts (like Symbola)
> > installed, but Emacs still shows some characters as boxes with hex
> > code, instead of using Symbola.
> 
> This doesn't seem to be a problem in Ubuntu and/or Fedora (the systems I 
> typically use)

I think you are judging by characters that are sufficiently well
covered.  Try Emoji (something in the range U+1F600 to U+1F64F), which
was the trigger for this bug and my changes, or Enclosed Alphanumerics
(U+1F100 to U+1F1FFF), for example.  Or even Supplemental Punctuation
(u+2E00 to u+2E7F) or Currency Symbols (u+20A0 to u+20CF).  What I see
on my system is that several fonts claim coverage, but typically
support just a few characters, sometimes just one.  That's the problem
I was trying to avoid.

> and on these environments the cure seems to be worse than the
> disease.  Is there some way we can heuristically tell whether we're
> in an environment where glyphs are often not supported?

It's not a matter of the environment, it's the matter of which fonts
are installed.  And sometimes, more installed fonts is worse, because
some fonts claim support for Unicode ranges where their actual
coverage is very scarce, tempting Emacs to try using them instead of
better ones.

> For example, can we convert a sample glyph or two to a bitmap and
> see whether it looks like a boxed hex code?  (Just thinking out
> loud.)

That's not necessary: Emacs can know whether a font supports a
character without any analysis of the resultant bitmap (see
font_has_char).  The problem is that Emacs tries very hard not to open
a font, and does all its font search without actually opening any
fonts.  By contrast, what you suggest is only possible once we open a
font.

I think the basic idea to augment the default fontset is correct, it
just needs to be fine-tuned to not mess up users' most popular setups.

> > I don't think it's a good idea to
> > go back to the previous arrangement where any font that claimed
> > iso10646-1 support would be considered as covering symbols and
> > punctuation well, because that means restoring the problems I tried to
> > fix in the first place.
> 
> In that case I don't understand why emacs -Q -font 
> -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso10646-1 works now.  
> If 
> Emacs is supposed to prefer Symbola to other fonts when displaying symbols, 
> why 
> isn't it using Symbola in this case?

Because -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso10646-1
is different from
-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-1, I
think.  The current code on master already does use the default font
if it supports these characters, but
-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-1
doesn't, evidently.  So Emacs tries to look for another font.  Before
my changes it would find
-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso10646-1;
after them, it tries Symbola first.