qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [Qemu-block] Migration sometimes fails with IDE and Qem


From: Peter Lieven
Subject: Re: [Qemu-devel] [Qemu-block] Migration sometimes fails with IDE and Qemu 2.2.1
Date: Tue, 07 Apr 2015 21:02:34 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0

Am 07.04.2015 um 20:56 schrieb John Snow:
>
>
> On 04/07/2015 02:44 PM, Peter Lieven wrote:
>> Am 07.04.2015 um 17:29 schrieb Dr. David Alan Gilbert:
>>> * Peter Lieven (address@hidden) wrote:
>>>> Hi David,
>>>>
>>>> Am 07.04.2015 um 10:43 schrieb Dr. David Alan Gilbert:
>>>>>>>> Any particular workload or reproducer?
>>>>>>> Workload is almost zero. I try to figure out if there is a way to 
>>>>>>> trigger it.
>>>>>>>
>>>>>>> Maybe playing a role: Machine type is -M pc1.2 and we set -kvmclock as
>>>>>>> CPU flag since kvmclock seemed to be quite buggy in 2.6.16...
>>>>>>>
>>>>>>> Exact cmdline is:
>>>>>>> /usr/bin/qemu-2.2.1  -enable-kvm  -M pc-1.2  -nodefaults -netdev 
>>>>>>> type=tap,id=guest2,script=no,downscript=no,ifname=tap2  -device 
>>>>>>> e1000,netdev=guest2,mac=52:54:00:ff:00:65 -drive 
>>>>>>> format=raw,file=iscsi://172.21.200.53/iqn.2001-05.com.equallogic:4-52aed6-88a7e99a4-d9e00040fdc509a3-XXX-hd0/0,if=ide,cache=writeback,aio=native
>>>>>>>   -serial null  -parallel null  -m 1024 -smp 
>>>>>>> 2,sockets=1,cores=2,threads=1  -monitor tcp:0:4003,server,nowait -vnc 
>>>>>>> :3 -qmp tcp:0:3003,server,nowait -name 'XXX' -boot 
>>>>>>> order=c,once=dc,menu=off  -drive 
>>>>>>> index=2,media=cdrom,if=ide,cache=unsafe,aio=native,readonly=on  -k de  
>>>>>>> -incoming tcp:0:5003  -pidfile /var/run/qemu/vm-146.pid  -mem-path 
>>>>>>> /hugepages  -mem-prealloc  -rtc base=utc -usb -usbdevice tablet 
>>>>>>> -no-hpet -vga cirrus  -cpu qemu64,-kvmclock
>>>>>>>
>>>>>>> Exact kernel is:
>>>>>>> 2.6.16.46-0.12-smp (i think this is SLES10 or sth.)
>>>>>>>
>>>>>>> The machine does not hang. It seems just I/O is hanging. So you can 
>>>>>>> type at the console or ping the system, but no longer login.
>>>>>>>
>>>>>>> Thank you,
>>>>>>> Peter
>>>>>> Interesting observation: Migrating the vServer again seems to fix to 
>>>>>> problem (at least in one case I could test just now).
>>>>>>
>>>>>> 2.6.8-24-smp is also affected.
>>>>> How often does it fail - you say 'sometimes' - is it a 1/10 or a 1/1000 ?
>>>> Its more often than 1/10 I would say.
>>> OK, that's not too bad - it's the 1/1000 that are really nasty to find.
>>> In your setup, how easy would it be for you to try :
>>>      with either 2.1 or current head?
>>>      with a newer machine-type?
>>>      without the cdrom?
>>
>> Its all possible. I can clone the system and try everything on my test 
>> systems. I hope
>> it reproduces there.
>>
>> Has the cdrom the power of taking down the bus?
>>
>> Peter
>>
>
> I don't know if CDROM could stall the entire bus, but I suspect the reason 
> for asking is this: dgilbert and I had tracked down a problem previously 
> where during migration, outstanding requests being handled by the ATAPI code 
> can get lost during migration if, for instance, the user has only prepared 
> the command (via bmdma) but has not yet written to the register to activate 
> the command yet.

That sounds like it could be related.

>
> So if something like this happens:
>
> - User writes to the ATA registers to program a command
> - Migration occurs
> - User writes to the BMDMA register to initiate the command
>
> We can lose some of the state and data of the request. David had checked in a 
> workaround for at least ATAPI that simply coaxes the guest OS into trying the 
> command again to unstick it.

Do you have the commit for me?

>
> I think we determined last time that we couldn't fix this problem without 
> changing the migration format, so we opted not to do it for 2.3. We had also 
> only noticed it with ATAPI drives, not HDDs, so a proper fix got kicked down 
> the road since we thought the workaround was sufficient.

Maybe normally we use virtio nowadays and maybe the new kernel implementation 
(libata /dev/sdX) can't get locked? What I do not understand is how a second 
migration can unlock from this state?

>
> IIRC our success rate with reproducing it was something on the order of 1/50, 
> too.
>
> If you can reproduce it without a CDROM but using the BMDMA interface, that's 
> a good data point. If you can't reproduce it using the ISA interface, that's 
> a phenomenal data point and implicates BMDMA pretty heavily.

To be 100% sure we are talking about the same? How would I use the ISA and how 
would I use the BMDMA interface?

Thanks,
Peter



reply via email to

[Prev in Thread] Current Thread [Next in Thread]