bug#613: 23.0.60; Unicode display problems

[sand]

Kenichi Handa writes:
> In article <18613.24465.897881.636770@priss.frightenedpiglet.com>, 
> sand@blarg.net writes:
> 
> > I'm seeing a complete regression.  I rebuilt from CVS HEAD, confirmed
> > that your change is in fontset.el, and Emacs picks AR PL ShanHeiSun
> > for open single quotes.  Putting the script definitions back in fixes
> > the problem.  I have pasted "good" and "bad" font log blocks to the
> > end of this message.  Those logs are particularly for Neep Alt, but
> > the Misc Fixed ISO10646 fonts have the same problem.
> 
> [...]
> > Unsuccessful load of LEFT SINGLE QUOTE with new definition:
> 
> > font for: (8216)
> > ftfont-list: -jmk-Neep Alt-*-iso10646-1
> [...]
> >   -jmk-Neep Alt-normal-normal-normal-*-20-*-c-*-iso10646-1
> [...]
> >   -jmk-Neep Alt-normal-normal-normal-*-20-*-c-*-iso10646-1
> 
> Don't you happen to have two sets of "Neep Alt" fonts in the
> fontconfig's fontpath; one set supports LEFT SINGLE QUOTE,
> and the other doesn't?

Neep Alt was originally designed for ISO8859-1 encodings, and my
installation still has iso18859-1 fonts that definitely lack LEFT
SINGLE QUOTE.  However, I constructed additional fonts to support
ISO10646-1, by extracting the Misc Fixed glyphs outside of ISO8859-1,
and appending them to the Neep Alt font definitions.  Those fonts are
defined to have the iso10646-1 encoding.

When I first noticed the font display problem a couple of months ago,
I traced the font code in the debugger and found that Emacs considered
the ISO8859-1 fonts to be valid matches for LEFT SINGLE QUOTE and
similar codepoints.  It would:

  * load the ISO8859-1 font, rather than the ISO10646-1 font,
  * notice that LEFT SINGLE QUOTE was missing, and then
  * start looking in other font families for the codepoint.

Adding LEFT SINGLE QUOTE and a couple of others as "symbol"
representatives let Emacs filter out the ISO8859-1 fonts earlier in
the code path, leaving the ISO10646-1 fonts for Emacs to use.  The
Misc Fixed fonts had the same problem, as they also included ISO8859-1
and ISO10646-1 encodings, and this solution worked for them as well.

One approach that we might consider is to change the C code, so that
any ISO8859-1 font is immediately skipped when the codepoint is 256 or
greater.  Or we could change the font loading code to try multiple
encodings within a family before dropping back to other families.

Derek