[ft-devel] avoiding a longjmp crash in ftgrays.c

[Tom Bishop, Wenlin Institute]

Dear FreeTypers,

This was happening on MS-Windows 7/XP/Vista when trying to display a large, 
intricate character:

Program received signal SIGSEGV, Segmentation fault.
0x77c337c9 in msvcrt!_abnormal_termination ()
   from C:\WINDOWS\system32\msvcrt.dll
(gdb) bt
#0 0x77c337c9 in msvcrt!_abnormal_termination ()
   from C:\WINDOWS\system32\msvcrt.dll
#1 0x77c37856 in strerror () from C:\WINDOWS\system32\msvcrt.dll
#2 0x77c34e9a in msvcrt!longjmp () from C:\WINDOWS\system32\msvcrt.dll
#3 0x00000000 in ?? ()

In compliance with Murphy's Law, the crash would not occur if the program was 
compiled with debugging symbols and stack frames. Fortunately, "longjmp" was a 
clue, and there are only two places where FreeType (version 2.4.4) has longjmp: 
ftobjs.c and ftgrays.c. Revising ftgrays.c to avoid longjmp solved the problem. 
Here are the changes:

(1) Under "typedef struct  TWorker_", replace jump_buffer:

#ifndef FT_AVOID_LONGJMP
#define FT_AVOID_LONGJMP 1
#endif
#if FT_AVOID_LONGJMP
    int  outline_decompose_error;
#else
    ft_jmp_buf  jump_buffer;
#endif

(2) In gray_find_cell, replace ft_longjmp:

    if ( ras.num_cells >= ras.max_cells )
#if FT_AVOID_LONGJMP
    {
      ras.outline_decompose_error = ErrRaster_Memory_Overflow;
      return NULL;
    }
#else
      ft_longjmp( ras.jump_buffer, 1 );
#endif

(3) In gray_record_cell, after calling gray_find_cell, insert:

#if FT_AVOID_LONGJMP
      if ( cell == NULL )
        return;
#endif

(4) In gray_move_to, after calling gray_record_cell, insert:

#if FT_AVOID_LONGJMP
    if ( ras.outline_decompose_error != 0 )
      return ras.outline_decompose_error;
#endif

(5) In gray_convert_glyph_inner, replace ft_setjmp:

#if FT_AVOID_LONGJMP
    ras.outline_decompose_error = 0;
    error = FT_Outline_Decompose( &ras.outline, &func_interface, &ras );
    gray_record_cell( RAS_VAR );
    if ( ras.outline_decompose_error != 0 )
    {
      error = ras.outline_decompose_error;
      ras.outline_decompose_error = 0;
    }
#else
    if ( ft_setjmp( ras.jump_buffer ) == 0 )
    {
      error = FT_Outline_Decompose( &ras.outline, &func_interface, &ras );
      gray_record_cell( RAS_VAR );
    }
    else
      error = ErrRaster_Memory_Overflow;
#endif

I tested many times, with two versions of MinGW GCC (4.5 and 3.4.5), that the 
crash occurs when FT_AVOID_LONGJMP is 0 and not when it is 1. With GCC 4.5, the 
crash only seems to occur when compiled with -fomit-frame-pointer, but with GCC 
3.4.5 it also occurred with -fno-omit-frame-pointer. The crash originally 
occurred on someone else's computer, and I was able to reproduce it on my own 
computer. The Mac OS X version of the same program didn't crash.

I don't understand why longjmp causes SIGSEGV, or where the fault lies: with my 
program, or MS-Windows, or MinGW GCC, or FreeType. (I'd like to write a short 
program that illustrates the crash; unfortunately I can't afford the time right 
now.)

I don't understand the significance of ErrRaster_Memory_Overflow. Characters 
seem to be displayed correctly even when ErrRaster_Memory_Overflow occurs, and 
the machine has gigabytes of RAM to spare.

Maybe setjmp/longjmp should be avoided, at least when the alternative only 
requires eight additional lines of code, as is the case here. The above changes 
could be made to ftgrays.c, though FT_AVOID_LONGJMP might be 0 by default at 
least until the issue has been studied and tested further, ideally by people 
who understand ftgrays.c and FT_Outline_Decompose (in ftoutln.c) better.

I'm nervous that the other use of longjmp, by ft_validator_error (in ftobjs.c) 
could cause similar nightmares-before-Christmas. ft_validator_error is only 
used disguised as macros FT_INVALID and FT_INVALID_, defined in ftvalid.h, 
gxvcommn.h, and otvcommn.h. ftvalid.h contains this comment:

  /* Sets the error field in a validator, then calls `longjmp' to return */
  /* to high-level caller.  Using `setjmp/longjmp' avoids many stupid    */
  /* error checks within the validation routines.                        */

Personally, I'd feel safer with "many stupid error checks" than with longjmp.

Microsoft says, "Do not use setjmp and longjmp in C++ programs..." 
(<http://msdn.microsoft.com/en-us/library/yz2ez4as.aspx>). My program is plain 
C, but it includes some of ICU, part of which is C++; maybe that's a factor.

Merry Christmas!

Tom

文林 Wenlin Institute, Inc.        Software for Learning Chinese
E-mail: address@hidden     Web: http://www.wenlin.com
Telephone: 1-877-4-WENLIN (1-877-493-6546)
☯