[Qemu-discuss] ASan'ed binaries start up very slow under qemu-aarch64.

[Maxim Ostapenko]

Hi!

When testing AddressSanitizer tool for AArch64 under qemu-aarch64 (user 
mode), I found out that even trivial helloworld apps start extremely 
slow (~2 seconds). I've investigated this a bit and noticed that QEMU 
actually freezes in large mmaps and subsequent reads of /proc/self/maps. 
Here a scenario I observed:

1) AddressSanitizer mmaps quite large regions of memory for redzones and 
shadow gap. In particular, for 39-bit AS it mmapes:

|| `[0x1400000000, 0x1fffffffff]` || HighShadow || - 48 Gb
|| `[0x1200000000, 0x13ffffffff]` || ShadowGap  || - 8 Gb
|| `[0x1000000000, 0x11ffffffff]` || LowShadow  || - 4 Gb

2) In QEMU, page_set_flags is called for these ranges. It cuts given 
range to individual pages and sets flags for them.  Given the page size 
is 4 Kb, for 8 Gb range we have 2097152 iterations and for 48 Gb 
12582912 iterations in inner loop. This is obviously a performance 
bottleneck.

3) Same issue may happen when ASan tries to read /proc/self/map later in 
page_check_range function, after it already mmaped HighShadow, ShadowGap 
and LowShadow regions.

Could someone help me, how can I mitigate this performance issue? Do we 
really need to set flags to each page on entire (quite big) memory region?

Thanks,
-Maxim