[ft-devel] gray_render_scanline: very hot when fuzzing

[Kostya Serebryany]

Hi, 

When fuzzing freetype (https://github.com/kcc/libfuzzer-example/wiki/FreeType-Fuzzer-Bot)

50% of time is spent in a single function: gray_render_scanline.

This is the profile when running this function on a large corpus of tests:

https://github.com/kcc/libfuzzer-example/blob/master/freetype-experiment/freetype2_fuzzer.cc

 51.76%  repro_orig  repro_orig              [.] gray_render_scanline

 18.07%  repro_orig  [kernel.kallsyms]       [k] 0xffffffff8104f45a

 14.22%  repro_orig  repro_orig              [.] gray_set_cell

  6.39%  repro_orig  libc-2.19.so            [.] memset

  5.30%  repro_orig  repro_orig              [.] gray_render_line

  1.03%  repro_orig  repro_orig              [.] tt_cmap4_char_next

  0.76%  repro_orig  repro_orig              [.] tt_cmap4_next

Is it a skew in my test corpus, 

or gray_render_scanline is expected to be the hottest function?

All the cycles are coming from a single DIV instruction in src/smooth/ftgrays.c: 

       │         FT_DIV_MOD( TCoord, p, dx, delta, mod );

  0.54 │ 82:   mov    %rbp,%rax

  0.19 │       cqto

  1.04 │       idivq  0x18(%rsp)

 58.07 │       test   %rdx,%rdx

Is this something you would be able to fix? 

This is probably not a bug, but anything that can speed up 

fuzzing improves our chances to catch real bugs. 

Thanks, 

--kcc