[ft] Fwd: Re: Texts Rasterization Exposures

[David Turner]

By the way, here's my answer to Maxim's questions


----- Original message -----
From: "David Turner" <address@hidden>
To: "Maxim Shemanarev" <address@hidden>, address@hidden
Date: Tue, 10 Jul 2007 14:58:28 +0200
Subject: Re: Texts Rasterization Exposures

Hello Maxim,

On Sun, 8 Jul 2007 16:20:50 -0400, "Maxim Shemanarev" <address@hidden> said:
> Hi David,
> 
> I'm the author of the Anti-Grain Geometry project, and we have a contact 
> with you about 3 years ago. 

Yes, I remember pretty well. Good to be in contact again..

> I finally tried to summarize my experience and observations concerning the
> situation with text rasterization in Windows and Linux. The article also 
> contains demo applications to play with my method of RGB sub-pixel text
> rendering. I admit some statements may  sound questionable, so, I appreciate
> any comments, criticism, and  discussions. So, here it is:
> http://antigrain.com/research/font_rasterization/index.html
> 
> It would be very interesting to learn about your opinion. The simple method 
> I propose may be of potential interest in the GNU/Linux community.
> 
ok, I've read your page, and here are my remarks, in no relevant order:

- I think you're a bit harsh on Microsoft, most of the choices they've made are 
due
  to backwards compatibility. *Many* applications assume integer advance widths
  and caret position. they wouldn't work correctly with sub-pixel positioning 
and
  lack of hinting. 

  Apple didn't have this problem when they wrote OS X, because they didn't 
intend
  to support old applications except in a sort of virtualized way, so they could
  start from a fresh codebase and a completely redesigned text subsystem.

  And the same problem will happen on Linux if you want to implement sub-pixel
  positioning, because most of the GUI toolkits out there cannot deal with it 
correctly
  (and due to the way toolkits are highly fragmented, the problem isn't going 
to go
  away easily). And some applications will not work correctly anymore if you 
change
  the toolkit. Backwards compatibility is a bitch :-)

- LCD rendering introduces its own sort of challenges. For example, the RGB 
filtering
  tends to create wider bitmaps. What this means is that coding something like 
a terminal
  correctly becomes more challenging due to overlapping glyph boxes which do 
not happen
  normally with monospaced fonts. that's exactly why the libXft and Cairo 
default LCD filters
  only touch intra-pixel components: Keith Packard wanted to avoid to deal with 
this problem,
  or it would have needed changes in xterm and others, and he didn't want to do 
that.

  Funnily, I've provided patches for better filtering a long time ago (see the 
dates at:
  http://david.freetype.org/lcd for example), and this doesn't seem to create 
much problems
  in practice, except for the occasionnal color fringe on the borders of 
selected text and
  stuff like that...

  As a related note, some Windows editors (e.g. Ultra-Edit) even have special 
options to
  deal with ClearType (which in XP, still uses hinting and integer advances) 
because of this.

- Windows Vista supports sub-pixel positioning just fine (even without 
ClearType), but
  this is "reserved" to WPF applications only. Unsurprisingly, because the 
framework was
  designed with device-independence from the start. this also explains why you 
*cannot*
  disable anti-aliasing in WPF applications by the way.

- Microsoft Word is known to use font metrics in a weird way. Basically, it 
uses the
  hinted metrics corresponding to your printer's resolution to compute text 
layout.
  I'm not making this up. When it displays text on your page, it places it by 
scaling
  the printer metrics to the screen, which introduces all sorts of irregular 
spacing.

  of course, the really idiotic thing is that if you change your printer 
setting, it will
  reflow the document (and if you don't have any printer, it chooses a default 
resolution,
  which I don't remember here) in a different way.

  that's *very* stupid now, but I guess they did it for Word 1.0, where it was 
better to
  produce text that would fit well on a dotted daisy printer, which required 
very strong
  hinting to produce good results... Again, I smell backwards compatibility.

  However, you should *never* use Word as a comparison for text rendering, 
because this
  issue makes any of your comments pointless, unless you know exactly what 
you're speaking
  of.

- Something you may not know is that Adobe Acrobat Reader does weird things to 
font
  sometimes. It likes to slightly expand/contract glyph shapes to make them 
better fit
  the total paragraph width (not counting inter-word spacing). this is all 
pretty advanced
  stuff, but not always suitable for general application frameworks (it may 
also explain
  why text rendering is so slow in this program as well).

  Again, be careful when you use this program for comparison purposes...

- With FreeType's auto-hinter, it's possible to "correct" the inter-character 
spacing to
  better reflect the original glyph shapes. this process is called 
"auto-kerning" or
  "device-kerning", and consists of looking at the space between two glyphs 
before and
  after hinting. If the difference is too wide or too short, you can add or 
remove a
  spacing pixel for better results.

  I've added the "ftdiff" program to the ft2demos package to demonstrate this, 
also look at
  the following pictures:

     http://david.freetype.org/freetype/why-deltas-matter-1.png
     http://david.freetype.org/freetype/why-deltas-matter-2.png
     http://david.freetype.org/freetype/why-deltas-matter-3.png

  the left column is unhinted and sub-pixel positioned text.
  the middle column is normal hinted text with auto-kerning enabled
  the right column is normal hinted text, without auto-kerning

  auto-kerning is pretty simple to implement, it boils down to using the 
"rb_delta" and
  "lb_delta" returned by the auto-hinter.  Look at the end of the following 
section:

    
http://www.freetype.org/freetype2/docs/reference/ft2-base_interface.html#FT_GlyphSlotRec

  the algorithm is simply:

=======================================================================
    FT_Pos  origin_x       = 0;                                      
    FT_Pos  prev_rsb_delta = 0;                                      
                                                                   
                                                                   
    for all glyphs do                                                
      <compute kern between current and previous glyph and add it to 
       `origin_x'>                                                   
                                                                   
      <load glyph with `FT_Load_Glyph'>                              
                                                                   
      if ( prev_rsb_delta - face->glyph->lsb_delta >= 32 )           
        origin_x -= 64;                                              
      else if ( prev_rsb_delta - face->glyph->lsb_delta < -32 )      
        origin_x += 64;                                              
                                                                   
      prev_rsb_delta = face->glyph->rsb_delta;                       
                                                                   
      <save glyph image, or render glyph, or ...>                    
                                                                   
      origin_x += face->glyph->advance.x;                            
    endfor                             
==========================================================================

  and since the distortions are always very small, you can also speed it up
  by removing all the tests with:

       delta = prev_rsb_delta - face->glyph->lsb_delta;
       origin_x += (delta + 32) & ~63;

  (and of course, the rsb_delta and lsb_delta are always 0 for monospaced 
fonts).

  And yes, this has been available from FreeType since quite some time now :-)

- the trick of multiplying the Y resolution by 100, then scaling down the 
result is not
  very elegant. You'd better load the glyphs twice (once hinted, the other 
not), and merge
  the result (take the X values from the hinted result, the Y values from the 
unhinted ones)

- what surprises the most is that you don't seem to be aware of FreeType's 
capabilities. What
  you're asking for (hinting only in the vertical direction) *already* exists 
and is named
  "light hinting", in FreeType parlance. Most Linux distributions already 
implement something that
  is very very near what you're asking: go to the Gnome Font Control Panel, 
select "Advanced",
  then select "Light hinting" and "Subpixel Rendering".

  on recent distros, this will use the light hinter + LCD filtering. the only 
difference to your scheme
  is that sub-pixel glyph positioning is not implemented. However, this is a 
problem of the frameworks
  on top of FreeType, not the font engine itself (you can trivially translate 
the result of the light
  hinter yourself to achieve sub-pixel positioning). The result is still quite 
good in my opinion.
  For example, see http://david.freetype.org/lcd/lcd-filter-1.png, that's what 
I have on one of my
  desktop, not some fancy mockup.

  All other things you ask from the font engine have been available from a long 
time (the real linearly
  scaled advanced width, the linearly scaled kerning values, etc...). Please 
consult the documentation,
  I think you'll find it interesting.

- nonetheless, there is still a lot of work to do on top of the font engine to 
make text really better.
  If I had the time to start a "next-generation text rendering" initiative for 
Linux, I would probably
  list it as follows:

     - add support for sub-pixel positioning to most text layout toolkits used 
by a typical Linux
       desktop. this means modifying Pango, LibXft, Qt, Cairo, at the very 
least. Probably also some
       other libraries like poppler. I'm probably forgetting a lot. Things like 
Sun's JavaVM also
       perform their own font rendering and do not respect standard setting. 
this is gross and only
       leads to inconsistencies that make users cry.

     - add support for kerning and auto-kerning to the same toolkits (some 
support hinted kerning,
       which isn't very useful). auto-kerning is only useful when using 
hinting, ignore it otherwise.

     - LCD filtering/rendering is currently implemented in several libraries 
(libXft and Cairo), and
       it would be better if it was located in a single library. Since there 
are patent issues about
       it (see http://david.freetype.org/cleartype-patents.html), I would 
suggest placing this code
       in FreeType itself. Just like the TrueType native interpreter, it would 
be disabled by default
       builds of the library, but could be enabled by recompiling a single 
package.

       oh, and allow for better filtering than the current default in the 
upstream libraries.

     - implement gamma-correct alpha-blending when displaying text.
           basically needs changes to the XServer implementation, and probably 
to the XRender protocol
           to be done 100% correctly. So it's both hard and *very* difficult to 
get accepted/implemented.
           Can be approximated by clients otherwise, at the risk of exploding 
glyph cache footprint and 
           X11 traffic :-(

      - the current font preferences dialog in Gnome/KDE/Wathever are also a 
joke, they expose settings
        that are too obscure to most users. I would re-design them to present 
something more understandable
        to ordinary people and hackers alike.

  Alas, I lack the time and motivation to do all this work. If you feel like 
it, feel free to start such
  an initiative (find a good name first, and please avoid stupid acronyms). I 
have provided many patches
  in the past that are mostly ignored by the corresponding library's 
maintainers (but are surprisingly
  included by distribution package managers), and I'm quite tired of this.

Best Regards,

- David Turner
- The FreeType Project  (www.freetype.org)

> McSeem