Re: [Qemu-devel] Regression with windows 7 VMs and VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1)

[Denis V. Lunev]

On 05/15/2016 02:13 AM, Thomas Lamprecht wrote:
> Hi all,
>
> I recently ran into Problems when trying to install some Windows VMs
> this was after an update to QEMU 2.5.1.1, the VM shows Windows loading
> files for the installation, then the "Starting Windows" screen appears
> here it hangs and never continues.
>
> Changing the "-vga" option to cirrus solves this, the installation can
> proceed and finish. When changing back to std (or also qxl, vmware) the
> installed VM also hangs on the "Starting Windows" screen while qemu
> showing a little but no excessive load.
>
> This phenomena appears also with QEMU 2.6.0 but not with 2.6.0-rc4, a
> git bisect shows fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 (vga: make
> sure vga register setup for vbe stays intact (CVE-2016-3712)) as the
> culprit for this regression, as its a fix for a DoS its not an option to
> just revert it, I guess.
> The (short) bisect log is:
>
> git bisect start
> # bad: [bfc766d38e1fae5767d43845c15c79ac8fa6d6af] Update version for v2.6.0 
> release
> git bisect bad bfc766d38e1fae5767d43845c15c79ac8fa6d6af
> # good: [975eb6a547f809608ccb08c221552f666611af25] Update version for 
> v2.6.0-rc4 release
> git bisect good 975eb6a547f809608ccb08c221552f666611af25
> # good: [2068192dcccd8a80dddfcc8df6164cf9c26e0fc4] vga: update vga register 
> setup on vbe changes
> git bisect good 2068192dcccd8a80dddfcc8df6164cf9c26e0fc4
> # bad: [53db932604dfa7bb9241d132e0173894cf54261c] Merge remote-tracking branch 
> 'remotes/kraxel/tags/pull-vga-20160509-1' into staging
> git bisect bad 53db932604dfa7bb9241d132e0173894cf54261c
>
> I could reproduce that with QEMU 2.5.1 and QEMU 2.6 on a Debian derivate
> (Promox VE) with 4.4 Kernel and also with QEMU 2.6 on an Arch Linux
> System with a 4.5 Kernel, so it should not be host distro depended. Both
> machines have Intel x86_64 processors.
> The problem should be reproducible with said Versions or a build from
> git including the above mentioned commit (fd3c136) by starting a VM with
> an Windows 7 ISO, e.g.:
>
> Hanging installation
> ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024
>
> Working installation:
> ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 -vga cirrus
>
> Noteworthy may be that Windows 10 is working, I do not had time to get
> other Windows versions and test them, I'll do that as soon as possible.
> Various Linux system also seems to work fine, at least I did not ran
> into an issue there yet.
>
> I also tried testing with SeaBIOS and OVMF, as initially I had no idea
> what broke, both lead to the same result - without the CVE-2016-3712 fix
> they both work, with not.
> Further, KVM enabled and disabled does not make any difference.
>
> If I can take any further step, e.g. open a bug report at another place
> or help with testing I'd glad to do so.
>
> best regards,
> Thomas
we have (may be) similar stuff with win2k3. With these patches merged
we have win2k3 crashed with the bug check below.
We are using qemu-kvm-2.3.0-31.2.13

There are a bunch of VMs in this state and semi-reliable reproduction.

At the first glance here this looks like return of
https://bugzilla.redhat.com/show_bug.cgi?id=693993

Den

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

THREAD_STUCK_IN_DEVICE_DRIVER (ea)
The device driver is spinning in an infinite loop, most likely waiting for
hardware to become idle. This usually indicates problem with the hardware
itself or with the device driver programming the hardware incorrectly.
If the kernel debugger is connected and running when watchdog detects a
timeout condition then DbgBreakPoint() will be called instead of KeBugCheckEx()
and detailed message including bugcheck arguments will be printed to the
debugger. This way we can identify an offending thread, set breakpoints in it,
and hit go to return to the spinning code to debug it further. Because
KeBugCheckEx() is not called the .bugcheck directive will not return bugcheck
information in this case. The arguments are already printed out to the kernel
debugger. You can also retrieve them from a global variable via
"dd watchdog!g_WdBugCheckData l5" (use dq on NT64).
On MP machines (OS builds <= 3790) it is possible to hit a timeout when the 
spinning thread is
interrupted by hardware interrupt and ISR or DPC routine is running at the time
of the bugcheck (this is because the timeout's work item can be delivered and
handled on the second CPU and the same time). If this is the case you will have
to look deeper at the offending thread's stack (e.g. using dds) to determine
spinning code which caused the timeout to occur.
Arguments:
Arg1: fffffadfe6fd3bf0, Pointer to a stuck thread object.  Do .thread then kb 
on it to find
        the hung location.
Arg2: fffffadfe6f45f00, Pointer to a DEFERRED_WATCHDOG object.
Arg3: fffffadfe7030350, Pointer to offending driver name.
Arg4: 0000000000000010, Number of times this error occurred.  If a debugger is 
attached,
        this error is not always fatal -- see DESCRIPTION below.  On the
        blue screen, this will always equal 1.

Debugging Details:
------------------

ERROR - could not read driver name for bugcheck parameter 3


FAULTING_THREAD:  fffffadfe6fd3bf0

FAULTING_IP:
vga!bEnableHardware+89
fffff97f`ff489609 85c0            test    eax,eax

IMAGE_NAME:  vga.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  42438b7b

MODULE_NAME: vga

FAULTING_MODULE: fffff97fff487000 vga

DEFAULT_BUCKET_ID:  GRAPHICS_DRIVER_FAULT

BUGCHECK_STR:  0xEA

PROCESS_NAME:  csrss.exe

CURRENT_IRQL:  1

LAST_CONTROL_TRANSFER:  from fffffadfe2fe3f03 to fffff8000102e950

STACK_TEXT:
fffffadf`e419c428 fffffadf`e2fe3f03 : 00000000`000000ea fffffadf`e6fd3bf0 
fffffadf`e6f45f00 fffffadf`e49acae0 : nt!KeBugCheckEx
fffffadf`e419c430 fffff800`01027f81 : fffffadf`e419cdd8 fffffadf`e419cdd4 
fffffadf`e70e8bf0 fffffadf`e3c9a158 : VIDEOPRT!WdpKernelApc+0x303
fffffadf`e419c9d0 fffff800`01027b8d : fffff800`00828cd0 00000000`00000000 
00000000`00000000 fffff800`00828cd0 : nt!KiDeliverApc+0x215
fffffadf`e419ca70 fffff800`00815c54 : 00000000`00000000 fffff800`00815d02 
00000000`00000010 00000000`00000246 : nt!KiApcInterrupt+0xdd
fffffadf`e419cc00 fffff800`0081b36a : 00000000`00000010 fffff800`00816627 
00000000`00000010 00000000`00000282 : hal!x86BiosWriteIoSpace+0xc4
fffffadf`e419cc40 fffff800`00816683 : 00000000`001a0018 fffff800`00828cd0 
00000000`00000000 fffff800`00800000 : hal!XmOutsOp+0xda
fffffadf`e419cc70 fffff800`0080e189 : fffff800`00828cd0 fffff800`01028090 
00000000`00000000 fffffadf`e419cce0 : hal!XmEmulateStream+0x133
fffffadf`e419ccc0 fffffadf`e2fed0ac : 00000000`00000000 fffff800`00000000 
fffffadf`e419cd00 00000000`00000000 : hal!HalCallBios+0x119
fffffadf`e419cd50 fffffadf`e45706db : 00000000`00000007 fffffadf`e76bf0c0 
fffffadf`e71ec5b8 fffffadf`e4575510 : VIDEOPRT!VideoPortInt10+0x9c
fffffadf`e419cda0 fffffadf`e45729ea : fffffadf`e741b400 fffffadf`e76bf0c0 
00000000`00000000 fffffadf`e419cf08 : vgapnp!VgaSetMode+0x13b
fffffadf`e419cdf0 fffffadf`e2ff7777 : 00000000`00000000 fffffadf`00000000 
fffffadf`e741b400 fffffadf`e71ec1d0 : vgapnp!VgaStartIO+0x29a
fffffadf`e419ce60 fffff97f`ff08676e : fffffadf`e71ec080 fffffadf`e741b3d0 
00000000`c0000002 fffffadf`e71ec080 : VIDEOPRT!pVideoPortDispatch+0xb67
fffffadf`e419cf80 fffff97f`ff086671 : 00000000`00000000 00000000`00000001 
000001e0`00000280 fffff97f`ff0ab843 : win32k!GreDeviceIoControl+0xdc
fffffadf`e419d040 fffff97f`ff489609 : 00000000`00000000 00000000`ffffffff 
00000000`00000001 fffff97f`ff0c66f9 : win32k!EngDeviceIoControl+0x37
fffffadf`e419d090 fffff97f`ff489304 : 00000000`00000010 00000000`00000000 
00000000`0b050033 fffffa80`007a5350 : vga!bEnableHardware+0x89
fffffadf`e419d160 fffff97f`ff004ec9 : 000001e0`00000280 fffffa80`00455620 
00000000`00000000 fffffa80`007a5350 : vga!DrvEnableSurface+0xc4
fffffadf`e419d1a0 fffff97f`ff2d3a51 : 00000000`64776447 fffffa80`008a2010 
fffffa80`0045e000 fffffa80`00955010 : win32k!WatchdogDrvEnableSurface+0x85
fffffadf`e419d1e0 fffff97f`ff0ac414 : fffffa80`0045e000 00000000`00000000 
00000000`00000000 fffffa80`0045e000 : win32k!PanEnableSurface+0x61
fffffadf`e419d2a0 fffff97f`ff007011 : fffffa80`0045e000 fffffa80`00949838 
fffffa80`0078d630 fffffa80`008a2010 : win32k!PDEVOBJ::bMakeSurface+0x43
fffffadf`e419d2f0 fffff97f`ff2304f1 : fffffa80`00303a30 fffffa80`00338b90 
fffffa80`0078d630 00000000`00000001 : win32k!hCreateHDEV+0x5fc
fffffadf`e419d3e0 fffff97f`ff00a7ac : 00000000`00000004 fffffadf`e419d490 
00000000`00000000 00000000`00000000 : win32k!DrvCreateMDEV+0xb7a
fffffadf`e419d740 fffff97f`ff008dcd : fffffadf`ffffffff 00000000`00000000 
00000000`01bc8000 00000000`00000000 : win32k!DrvChangeDisplaySettings+0x394
fffffadf`e419d970 fffff97f`ff00bb28 : 00000000`01bc8000 00000000`00000000 
00000000`00000003 00000000`00000000 : win32k!InitVideo+0x4c
fffffadf`e419d9e0 fffff97f`ff00b646 : 00000000`00000060 fffffadf`e419dcf0 
00000000`00000003 00000000`0000005c : win32k!UserInitialize+0x63c
fffffadf`e419dc20 fffff800`0102e3fd : fffffadf`000001e0 00000000`00000001 
fffffadf`e6fd3bf0 00000000`00000020 : win32k!NtUserInitialize+0x15f
fffffadf`e419dc70 000007ff`7c7de86a : 000007ff`7c7deaa9 00000000`001657f0 
00000000`001657f0 00000000`00000003 : nt!KiSystemServiceCopyEnd+0x3
00000000`0015fbc8 000007ff`7c7deaa9 : 00000000`001657f0 00000000`001657f0 
00000000`00000003 00000000`00000000 : winsrv!NtUserInitialize+0xa
00000000`0015fbd0 000007ff`7c5b3eb8 : 00000000`00000000 00000000`00000001 
00000000`001657f0 00000000`00000000 : winsrv!UserServerDllInitialization+0x229
00000000`0015fc50 000007ff`7c5b38c0 : 00000000`00163746 00000000`00000000 
00000000`0016372a 00000000`00163480 : CSRSRV!CsrLoadServerDll+0x228
00000000`0015fd10 000007ff`7c5b3a10 : 00000000`00163450 00000000`00000000 
00000000`00163450 00000000`00000000 : CSRSRV!CsrParseServerCommandLine+0x340
00000000`0015fee0 00000000`4a681121 : 00000000`00110000 00000000`0015ff88 
00000000`00163450 00000000`0000000a : CSRSRV!CsrServerInitialization+0xa0
00000000`0015ff10 00000000`4a6815bb : 00000000`0000000b 00000000`00000000 
00000000`0000010f 00000000`00163570 : csrss!main+0x41
00000000`0015ff60 00000000`00000000 : 00000000`00000001 00000000`00000000 
00000000`00000004 00000000`00000005 : csrss!NtProcessStartup+0x2fb