Hello developers,
When testing video on windows, I found the MSDrawDibDisplay video filter crash in sdk 5.2.x.
I'm using C# winform's picturebox to show video, this works in sdk 4.5.26. Since in 4.5.26, the value of Core.VideoDisplayFilter is MSDrawDibDisplay, I set
Core.VideoDisplayFilter = "MSDrawDibDisplay";
in sdk 5.2.x.
However, it crashes when I open a window to show video preview. The message is:
Unhandled exception at 0x79E9D613 (mediastreamer.dll) in CPFCcodSipPhone.exe.46856.dmp: 0xC0000005: Access violation writing location 0xFFFFFD00.
I dumped the ram and open in VS2022, and found the error line of code is line 934 (mov [edi], bx) of external\libyuv\source\scale_win.cc, function ScaleFilterCols_SSSE3():
// 2 Pixel loop.
xloop2:
movdqa xmm1, xmm2 // x0, x1 fractions.
paddd xmm2, xmm3 // x += dx
movzx ebx, word ptr [esi + eax] // 2 source x0 pixels
movd xmm0, ebx
psrlw xmm1, 9 // 7 bit fractions.
movzx ebx, word ptr [esi + edx] // 2 source x1 pixels
movd xmm4, ebx
pshufb xmm1, xmm5 // 0011
punpcklwd xmm0, xmm4
psubb xmm0, xmmword ptr kFsub80 // make pixels signed.
pxor xmm1, xmm6 // 0..7f and 7f..0
paddusb xmm1, xmm7 // +1 so 0..7f and 80..1
pmaddubsw xmm1, xmm0 // 16 bit, 2 pixels.
pextrw eax, xmm2, 1 // get x0 integer. next iteration.
pextrw edx, xmm2, 3 // get x1 integer. next iteration.
paddw xmm1, xmmword ptr kFadd40 // make pixels unsigned and round.
psrlw xmm1, 7 // 8.7 fixed point to low 8 bits.
packuswb xmm1, xmm1 // 8 bits, 2 pixels.
movd ebx, xmm1
mov [edi], bx (This line crashes)
lea edi, [edi + 2]
sub ecx, 2 // 2 pixels
jge xloop2
xloop29:
add ecx, 2 - 1
jl xloop99
The edi's value turns out to be 0xfffffd00, which is the address of function ScaleFilterCols_SSSE3. And the ecx's value shows that it is the second time of entering xloop2. This function's caller function is ScalePlaneBilinearDown() in external\libyuv\source\scale.cc and the parameter dst_ptr's value is ok.
I read the assembly code several times and could not understand why. Is it cause by the compiler's false optimizing?
This is the call stack of sdk when crashing:
> mediastreamer.dll!ScaleFilterCols_SSSE3(unsigned char * dst_ptr, const unsigned char * src_ptr, int dst_width, int x, int dx) Line 934 C++
mediastreamer.dll!ScalePlaneBilinearDown(int src_width, int src_height, int dst_width, int dst_height, int src_stride, int dst_stride, const unsigned char * src_ptr, unsigned char * dst_ptr, libyuv::FilterMode filtering) Line 1095 C++
mediastreamer.dll!ScalePlane(const unsigned char * src, int src_stride, int src_width, int src_height, unsigned char * dst, int dst_stride, int dst_width, int dst_height, libyuv::FilterMode filtering) Line 1635 C++
mediastreamer.dll!I420Scale(const unsigned char * src_y, int src_stride_y, const unsigned char * src_u, int src_stride_u, const unsigned char * src_v, int src_stride_v, int src_width, int src_height, unsigned char * dst_y, int dst_stride_y, unsigned char * dst_u, int dst_stride_u, unsigned char * dst_v, int dst_stride_v, int dst_width, int dst_height, libyuv::FilterMode filtering) Line 1759 C++
mediastreamer.dll!yuv_scale(_MSScalerContext * ctx, unsigned char * * src, int * src_strides, unsigned char * * dst, int * dst_strides) Line 506 C
mediastreamer.dll!yuv2rgb_process(Yuv2RgbCtx * ctx, _MSPicture * src, MSVideoSize dstsize, unsigned char mirroring) Line 104 C
mediastreamer.dll!dd_display_process(_MSFilter * f) Line 450 C
mediastreamer.dll!ms_filter_process(_MSFilter * f) Line 233 C
mediastreamer.dll!call_process(_MSFilter * f) Line 252 C
How can I fix this? Need your help. Thanks.