[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] Block Migration Assertion in qemu-kvm 1.2.0
|
From: |
Peter Lieven |
|
Subject: |
Re: [Qemu-devel] Block Migration Assertion in qemu-kvm 1.2.0 |
|
Date: |
Mon, 17 Sep 2012 22:12:31 +0200 |
|
User-agent: |
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111108 Thunderbird/3.1.16 |
On 09/17/12 10:41, Kevin Wolf wrote:
Am 16.09.2012 12:13, schrieb Peter Lieven:
Hi,
when trying to block migrate a VM from one node to another, the source
VM crashed with the following assertion:
block.c:3829: bdrv_set_in_use: Assertion `bs->in_use != in_use' failed.
Is this sth already addresses/known?
Not that I'm aware of, at least.
Block migration doesn't seem to check whether the device is already in
use, maybe this is the problem. Not sure why it would be in use, though,
and in my quick test it didn't crash.
So we need some more information: What's you command line, did you do
anything specific in the monitor with block devices, what does the
stacktrace look like, etc.?
kevin, it seems that i can very easily force a crash if I cancel a
running block migration.
here is a stack trace. i was not yet able to reproduce the asserts.
please note that all the new threads where created at the moment i
cancelled the migration.
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /usr/bin/nice...(no debugging symbols found)...done.
Starting program: /usr/bin/nice -n 1 /usr/bin/qemu-kvm-1.2.0 -net
tap,vlan=164,script=no,downscript=no,ifname=tap0 -net
nic,vlan=164,model=e1000,macaddr=52:54:00:ff:01:19 -drive
format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-fbef4e007-f0f0017cdc25055a-lieven-storage-migration-t-hd0,if=virtio,cache=none,aio=native
-m 2048 -smp 2,sockets=1,cores=2,threads=1 -monitor
tcp:0:4001,server,nowait -vnc :1 -qmp tcp:0:3001,server,nowait -name
lieven-storage-migration-test -boot order=dc,menu=off -k de -pidfile
/var/run/qemu/vm-254.pid -mem-path /hugepages -mem-prealloc -cpu
host,+x2apic,model_id=Intel\(R\)\ Xeon\(R\)\ CPU\ \ \ \ \ \ \ \ \ \ \
L5640\ \ @\ 2.27GHz,-tsc -rtc base=utc -usb -usbdevice tablet -no-hpet
-vga cirrus
process 29659 is executing new program:
/usr/local/qemu-kvm-1.2.0/bin/qemu-system-x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
165 block-migration.c: No such file or directory.
[New Thread 0x7ffff6163700 (LWP 29663)]
[New Thread 0x7ffff5962700 (LWP 29664)]
[New Thread 0x7fffefe97700 (LWP 29673)]
[New Thread 0x7fffef696700 (LWP 29674)]
[Thread 0x7fffefe97700 (LWP 29673) exited]
[New Thread 0x7fffefe97700 (LWP 29825)]
[Thread 0x7fffefe97700 (LWP 29825) exited]
[New Thread 0x7fffefe97700 (LWP 29969)]
[Thread 0x7fffefe97700 (LWP 29969) exited]
[New Thread 0x7fffefe97700 (LWP 30091)]
[New Thread 0x7fffee15c700 (LWP 30092)]
[New Thread 0x7fffecc47700 (LWP 30093)]
[New Thread 0x7fffdbfff700 (LWP 30094)]
[New Thread 0x7fffe33f3700 (LWP 30095)]
[New Thread 0x7fffe20e0700 (LWP 30096)]
[New Thread 0x7fffe18df700 (LWP 30097)]
[New Thread 0x7fffe10de700 (LWP 30098)]
[New Thread 0x7fffe08dd700 (LWP 30099)]
[New Thread 0x7fffdb7fe700 (LWP 30100)]
[New Thread 0x7fffdaffd700 (LWP 30101)]
[New Thread 0x7fffda7fc700 (LWP 30102)]
[New Thread 0x7fffd9ffb700 (LWP 30103)]
[New Thread 0x7fffd97fa700 (LWP 30104)]
[New Thread 0x7fffd8ff9700 (LWP 30105)]
[New Thread 0x7fffa7fff700 (LWP 30106)]
[New Thread 0x7fffa77fe700 (LWP 30107)]
[New Thread 0x7fffa6ffd700 (LWP 30108)]
[New Thread 0x7fffed853700 (LWP 30109)]
[New Thread 0x7fffa67fc700 (LWP 30110)]
[New Thread 0x7fffa5ffb700 (LWP 30111)]
[New Thread 0x7fffa57fa700 (LWP 30112)]
[New Thread 0x7fffa4ff9700 (LWP 30113)]
[New Thread 0x7fff83fff700 (LWP 30114)]
[New Thread 0x7fff837fe700 (LWP 30115)]
[New Thread 0x7fff82ffd700 (LWP 30116)]
[New Thread 0x7fff827fc700 (LWP 30117)]
[New Thread 0x7fff81ffb700 (LWP 30118)]
[New Thread 0x7fff817fa700 (LWP 30119)]
[New Thread 0x7fff80ff9700 (LWP 30120)]
[New Thread 0x7fff5ffff700 (LWP 30121)]
[New Thread 0x7fff5f7fe700 (LWP 30122)]
[New Thread 0x7fff5effd700 (LWP 30123)]
[New Thread 0x7fff5e7fc700 (LWP 30124)]
[New Thread 0x7fff5dffb700 (LWP 30125)]
[New Thread 0x7fff5d7fa700 (LWP 30126)]
[New Thread 0x7fff5cff9700 (LWP 30127)]
[New Thread 0x7fff3ffff700 (LWP 30128)]
[New Thread 0x7fff3f7fe700 (LWP 30129)]
[New Thread 0x7fff3effd700 (LWP 30130)]
[New Thread 0x7fff3e7fc700 (LWP 30131)]
[New Thread 0x7fff3dffb700 (LWP 30132)]
[New Thread 0x7fff3d7fa700 (LWP 30133)]
[New Thread 0x7fff3cff9700 (LWP 30134)]
[New Thread 0x7fff1ffff700 (LWP 30135)]
[New Thread 0x7fff1f7fe700 (LWP 30136)]
[New Thread 0x7fff1effd700 (LWP 30137)]
[New Thread 0x7fff1e7fc700 (LWP 30138)]
[New Thread 0x7fff1dffb700 (LWP 30139)]
[New Thread 0x7fff1d7fa700 (LWP 30140)]
[New Thread 0x7fff1cff9700 (LWP 30141)]
[New Thread 0x7ffefffff700 (LWP 30142)]
[New Thread 0x7ffeff7fe700 (LWP 30143)]
[New Thread 0x7ffefeffd700 (LWP 30144)]
[New Thread 0x7ffefe7fc700 (LWP 30145)]
[New Thread 0x7ffefdffb700 (LWP 30146)]
[New Thread 0x7ffefd7fa700 (LWP 30147)]
[New Thread 0x7ffefcff9700 (LWP 30148)]
[New Thread 0x7ffedffff700 (LWP 30149)]
[New Thread 0x7ffedf7fe700 (LWP 30150)]
[New Thread 0x7ffedeffd700 (LWP 30151)]
[New Thread 0x7ffede7fc700 (LWP 30152)]
[New Thread 0x7ffeddffb700 (LWP 30153)]
[New Thread 0x7ffedd7fa700 (LWP 30154)]
Program received signal SIGSEGV, Segmentation fault.
0x00005555555cd5f6 in bmds_set_aio_inflight (bmds=0x555556568280,
sector_num=2396160, nb_sectors=2048, set=0) at block-migration.c:165
(gdb) thread apply all bt full
Thread 71 (Thread 0x7ffedd7fa700 (LWP 30154)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffedd7f9eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 65877}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 70 (Thread 0x7ffeddffb700 (LWP 30153)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffeddffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 627866}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 69 (Thread 0x7ffede7fc700 (LWP 30152)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffede7fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 759753}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 68 (Thread 0x7ffedeffd700 (LWP 30151)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffedeffceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 644535}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 67 (Thread 0x7ffedf7fe700 (LWP 30150)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffedf7fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 712004}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 66 (Thread 0x7ffedffff700 (LWP 30149)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffedfffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 743448}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 65 (Thread 0x7ffefcff9700 (LWP 30148)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffefcff8eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555a4074a0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 754495}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 64 (Thread 0x7ffefd7fa700 (LWP 30147)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffefd7f9eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 762992}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 63 (Thread 0x7ffefdffb700 (LWP 30146)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffefdffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 28676}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 62 (Thread 0x7ffefe7fc700 (LWP 30145)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffefe7fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555a4074a0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 809680}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 61 (Thread 0x7ffefeffd700 (LWP 30144)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffefeffceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 810945}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 60 (Thread 0x7ffeff7fe700 (LWP 30143)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffeff7fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 33778}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 59 (Thread 0x7ffefffff700 (LWP 30142)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7ffeffffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 98618}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 58 (Thread 0x7fff1cff9700 (LWP 30141)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff1cff8eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 816786}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 57 (Thread 0x7fff1d7fa700 (LWP 30140)):
#0 0x00007ffff6971653 in pread64 () from
/lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573b0dd in handle_aiocb_rw_linear (aiocb=0x55555bc7eb20,
buf=0x7fff04000c00 "\326\246\002") at posix-aio-compat.c:224
offset = 0
len = 1048576
#2 0x000055555573b2c4 in handle_aiocb_rw (aiocb=0x55555bc7eb20)
at posix-aio-compat.c:293
nbytes = 93825003889184
buf = 0x7fff04000c00 "\326\246\002"
#3 0x000055555573b54c in aio_thread (unused=0x0) at posix-aio-compat.c:351
aiocb = 0x55555bc7eb20
ret = 0
tv = {tv_sec = 1347912395, tv_usec = 983439}
ts = {tv_sec = 1347912405, tv_nsec = 0}
#4 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 56 (Thread 0x7fff1dffb700 (LWP 30139)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff1dffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7f150
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 32930}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 55 (Thread 0x7fff1e7fc700 (LWP 30138)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff1e7fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 816746}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 54 (Thread 0x7fff1effd700 (LWP 30137)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff1effceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 42141}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 53 (Thread 0x7fff1f7fe700 (LWP 30136)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff1f7fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 121256}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 52 (Thread 0x7fff1ffff700 (LWP 30135)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff1fffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 48151}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 51 (Thread 0x7fff3cff9700 (LWP 30134)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff3cff8eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7e370
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 207541}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 50 (Thread 0x7fff3d7fa700 (LWP 30133)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff3d7f9eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 73822}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 49 (Thread 0x7fff3dffb700 (LWP 30132)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff3dffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 128860}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 48 (Thread 0x7fff3e7fc700 (LWP 30131)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff3e7fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 175835}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 47 (Thread 0x7fff3effd700 (LWP 30130)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff3effceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7e370
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 254947}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 46 (Thread 0x7fff3f7fe700 (LWP 30129)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff3f7fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 195999}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 45 (Thread 0x7fff3ffff700 (LWP 30128)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff3fffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 239779}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 44 (Thread 0x7fff5cff9700 (LWP 30127)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff5cff8eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 180411}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 43 (Thread 0x7fff5d7fa700 (LWP 30126)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff5d7f9eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 173028}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 42 (Thread 0x7fff5dffb700 (LWP 30125)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff5dffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 325936}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 41 (Thread 0x7fff5e7fc700 (LWP 30124)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff5e7fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7e370
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 185622}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 40 (Thread 0x7fff5effd700 (LWP 30123)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff5effceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7e370
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 221321}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 39 (Thread 0x7fff5f7fe700 (LWP 30122)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff5f7fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 258187}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 38 (Thread 0x7fff5ffff700 (LWP 30121)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff5fffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 346934}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 37 (Thread 0x7fff80ff9700 (LWP 30120)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff80ff8eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 282933}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 36 (Thread 0x7fff817fa700 (LWP 30119)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff817f9eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 330640}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 35 (Thread 0x7fff81ffb700 (LWP 30118)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff81ffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 330090}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 34 (Thread 0x7fff827fc700 (LWP 30117)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff827fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7f150
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 373531}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 33 (Thread 0x7fff82ffd700 (LWP 30116)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff82ffceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7e370
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 324347}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 32 (Thread 0x7fff837fe700 (LWP 30115)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff837fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 362610}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 31 (Thread 0x7fff83fff700 (LWP 30114)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fff83ffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 415042}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 30 (Thread 0x7fffa4ff9700 (LWP 30113)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffa4ff8eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7f150
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 342087}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 29 (Thread 0x7fffa57fa700 (LWP 30112)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffa57f9eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7f150
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 518047}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 28 (Thread 0x7fffa5ffb700 (LWP 30111)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffa5ffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 514193}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 27 (Thread 0x7fffa67fc700 (LWP 30110)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffa67fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 526408}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 26 (Thread 0x7fffed853700 (LWP 30109)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffed852eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 382861}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 25 (Thread 0x7fffa6ffd700 (LWP 30108)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffa6ffceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 398159}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 24 (Thread 0x7fffa77fe700 (LWP 30107)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffa77fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7e370
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 519253}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 23 (Thread 0x7fffa7fff700 (LWP 30106)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffa7ffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 624913}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 22 (Thread 0x7fffd8ff9700 (LWP 30105)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffd8ff8eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 560397}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 21 (Thread 0x7fffd97fa700 (LWP 30104)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffd97f9eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x5555565fe600
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 522208}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 20 (Thread 0x7fffd9ffb700 (LWP 30103)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffd9ffaeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7eb20
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 531203}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 19 (Thread 0x7fffda7fc700 (LWP 30102)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffda7fbeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 542959}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 18 (Thread 0x7fffdaffd700 (LWP 30101)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffdaffceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555a4074a0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 534592}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 17 (Thread 0x7fffdb7fe700 (LWP 30100)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffdb7fdeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 536674}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 16 (Thread 0x7fffe08dd700 (LWP 30099)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffe08dceb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555a4074a0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 629188}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 15 (Thread 0x7fffe10de700 (LWP 30098)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffe10ddeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7e370
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 40152}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 14 (Thread 0x7fffe18df700 (LWP 30097)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffe18deeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 650586}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 13 (Thread 0x7fffe20e0700 (LWP 30096)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffe20dfeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ed30
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 565139}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 12 (Thread 0x7fffe33f3700 (LWP 30095)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffe33f2eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 629929}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 11 (Thread 0x7fffdbfff700 (LWP 30094)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffdbffeeb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 592997}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 10 (Thread 0x7fffecc47700 (LWP 30093)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffecc46eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555bc7ef40
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 691042}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 9 (Thread 0x7fffee15c700 (LWP 30092)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffee15beb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x55555a4074a0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 710341}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 8 (Thread 0x7fffefe97700 (LWP 30091)):
#0 0x00007ffff696e0fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555573acf1 in cond_timedwait (cond=0x555556072260,
mutex=0x555556072220, ts=0x7fffefe96eb0) at posix-aio-compat.c:107
ret = 0
#2 0x000055555573b468 in aio_thread (unused=0x0) at posix-aio-compat.c:337
aiocb = 0x555559d064e0
ret = 0
tv = {tv_sec = 1347912396, tv_usec = 707609}
ts = {tv_sec = 1347912406, tv_nsec = 0}
#3 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 5 (Thread 0x7fffef696700 (LWP 29674)):
#0 0x00007ffff696dd84 in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x000055555575f7ae in qemu_cond_wait (cond=0x5555565b6c90,
mutex=0x5555565b6cc0) at qemu-thread-posix.c:113
err = 0
__func__ = "qemu_cond_wait"
#2 0x0000555555797c69 in vnc_worker_thread_loop (queue=0x5555565b6c90)
at ui/vnc-jobs.c:222
job = 0x5555565a8980
entry = 0x0
tmp = 0x0
vs = {csock = -1, ds = 0x55555653faf0, dirty = {{0, 0,
0} <repeats 2048 times>}, lossy_rect = 0x55555bc7ebb0,
vd = 0x7fffeffba010, need_update = 0, force_update = 0,
features = 0, absolute = 0, last_x = 0, last_y = 0,
client_width = 0, client_height = 0, share_mode = 0,
vnc_encoding = 0, major = 0, minor = 0, auth = 0,
challenge = '\000' <repeats 15 times>, info = 0x0, output = {
capacity = 3149825, offset = 3145744, buffer =
0x7fffee360010 ""},
input = {capacity = 0, offset = 0, buffer = 0x0},
write_pixels = 0x55555579a63d <vnc_write_pixels_copy>,
clientds = {
flags = 0 '\000', width = 1024, height = 768, linesize = 4096,
data = 0x7fffee967010 "\032\023*", pf = {bits_per_pixel =
32 ' ',
bytes_per_pixel = 4 '\004', depth = 24 '\030', rmask =
16711680,
gmask = 65280, bmask = 255, amask = 0, rshift = 16 '\020',
gshift = 8 '\b', bshift = 0 '\000', ashift = 24 '\030',
rmax = 255 '\377', gmax = 255 '\377', bmax = 255 '\377',
amax = 255 '\377', rbits = 8 '\b', gbits = 8 '\b',
bbits = 8 '\b', abits = 8 '\b'}}, audio_cap = 0x0, as = {
freq = 0, nchannels = 0, fmt = AUD_FMT_U8, endianness = 0},
read_handler = 0, read_handler_expect = 0,
modifiers_state = '\000' <repeats 255 times>, led = 0x0,
abort = false, output_mutex = {lock = {__data = {__lock = 0,
__count = 0, __owner = 0, __nusers = 0, __kind = 0,
__spins = 0, __list = {__prev = 0x0, __next = 0x0}},
__size = '\000' <repeats 39 times>, __align = 0}}, bh = 0x0,
jobs_buffer = {capacity = 0, offset = 0, buffer = 0x0}, tight
= {
type = 0, quality = 255 '\377', compression = 9 '\t',
pixel24 = 0 '\000', tight = {capacity = 0, offset = 0,
buffer = 0x0}, tmp = {capacity = 0, offset = 0, buffer =
0x0},
zlib = {capacity = 0, offset = 0, buffer = 0x0}, gradient = {
capacity = 0, offset = 0, buffer = 0x0}, levels = {0, 0,
0, 0},
stream = {{next_in = 0x0, avail_in = 0, total_in = 0,
next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0,
state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0,
data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0,
avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0,
total_out = 0, msg = 0x0, state = 0x0, zalloc = 0,
zfree = 0,
opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {
next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0,
avail_out = 0, total_out = 0, msg = 0x0, state = 0x0,
zalloc = 0, zfree = 0, opaque = 0x0, data_type = 0,
adler = 0,
reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0,
next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0,
state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0,
data_type = 0, adler = 0, reserved = 0}}}, zlib = {zlib
= {
capacity = 0, offset = 0, buffer = 0x0}, tmp = {capacity
= 0,
offset = 0, buffer = 0x0}, stream = {next_in = 0x0,
avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0,
total_out = 0, msg = 0x0, state = 0x0, zalloc = 0, zfree
= 0,
opaque = 0x0, data_type = 0, adler = 0, reserved = 0},
level = 0}, hextile = {
send_tile = 0x5555557849a0 <send_hextile_tile_32>}, zrle = {
type = 0, fb = {capacity = 0, offset = 0, buffer = 0x0},
zrle = {
capacity = 0, offset = 0, buffer = 0x0}, tmp = {capacity
= 0,
offset = 0, buffer = 0x0}, zlib = {capacity = 0, offset = 0,
buffer = 0x0}, stream = {next_in = 0x0, avail_in = 0,
total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0,
msg = 0x0, state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0,
data_type = 0, adler = 0, reserved = 0}, palette = {pool
= {{
idx = 0, color = 0, next = {le_next = 0x0,
le_prev = 0x0}} <repeats 256 times>}, size = 0, max
= 0,
bpp = 0, table = {{lh_first = 0x0} <repeats 256 times>}}},
zywrle = {buf = {0 <repeats 4090 times>, 128, 0, -157897112,
32767,
0, 0}}, mouse_mode_notifier = {notify = 0, node = {
le_next = 0x0, le_prev = 0x55555575f948}}, next = {
tqe_next = 0x0, tqe_prev = 0x5555565b6ce8}}
n_rectangles = 1
saved_offset = 2
#3 0x0000555555798192 in vnc_worker_thread (arg=0x5555565b6c90)
at ui/vnc-jobs.c:318
queue = 0x5555565b6c90
#4 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 3 (Thread 0x7ffff5962700 (LWP 29664)):
#0 0x00007ffff668fd27 in ioctl () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1 0x000055555580ebfa in kvm_vcpu_ioctl (env=0x555556543470, type=44672)
at /usr/src/qemu-kvm-1.2.0/kvm-all.c:1693
ret = -321675584
arg = 0x0
ap = {{gp_offset = 24, fp_offset = 48,
overflow_arg_area = 0x7ffff5961e70,
reg_save_area = 0x7ffff5961db0}}
#2 0x000055555580e653 in kvm_cpu_exec (env=0x555556543470)
at /usr/src/qemu-kvm-1.2.0/kvm-all.c:1580
run = 0x7ffff7fc3000
ret = 0
run_ret = 0
#3 0x00005555557a715b in qemu_kvm_cpu_thread_fn (arg=0x555556543470)
at /usr/src/qemu-kvm-1.2.0/cpus.c:757
env = 0x555556543470
cpu = 0x555556543430
r = 65536
#4 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 2 (Thread 0x7ffff6163700 (LWP 29663)):
#0 0x00007ffff668fd27 in ioctl () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1 0x000055555580ebfa in kvm_vcpu_ioctl (env=0x5555565267a0, type=44672)
at /usr/src/qemu-kvm-1.2.0/kvm-all.c:1693
ret = 32767
arg = 0x0
ap = {{gp_offset = 24, fp_offset = 48,
overflow_arg_area = 0x7ffff6162e70,
reg_save_area = 0x7ffff6162db0}}
#2 0x000055555580e653 in kvm_cpu_exec (env=0x5555565267a0)
at /usr/src/qemu-kvm-1.2.0/kvm-all.c:1580
run = 0x7ffff7fc6000
ret = 0
run_ret = 0
#3 0x00005555557a715b in qemu_kvm_cpu_thread_fn (arg=0x5555565267a0)
at /usr/src/qemu-kvm-1.2.0/cpus.c:757
env = 0x5555565267a0
cpu = 0x555556526760
r = 65536
#4 0x00007ffff6969e9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007ffff66974bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 1 (Thread 0x7ffff7fef700 (LWP 29659)):
#0 0x00005555555cd5f6 in bmds_set_aio_inflight (bmds=0x555556568280,
sector_num=2396160, nb_sectors=2048, set=0) at block-migration.c:165
start = 1170
end = 1170
val = 93824994379674
idx = 18
bit = 18
#1 0x00005555555cd7dc in blk_mig_read_cb (opaque=0x55555868e510, ret=0)
at block-migration.c:199
curr_time = 221018445336304
blk = 0x55555868e510
__PRETTY_FUNCTION__ = "blk_mig_read_cb"
#2 0x00005555555d755a in bdrv_co_em_bh (opaque=0x5555565fee50) at
block.c:3320
acb = 0x5555565fee50
#3 0x00005555555be8f7 in qemu_bh_poll () at async.c:71
bh = 0x5555565b6ea0
bhp = 0x7fffffffe170
next = 0x5555565a8980
ret = 1
nesting = 1
#4 0x00005555557280e6 in main_loop_wait (nonblocking=0) at main-loop.c:506
ret = 1
timeout = 4294967295
#5 0x000055555571ff9e in main_loop () at /usr/src/qemu-kvm-1.2.0/vl.c:1643
nonblocking = false
last_io = 1
#6 0x0000555555726b93 in main (argc=38, argv=0x7fffffffe5f8,
envp=0x7fffffffe730) at /usr/src/qemu-kvm-1.2.0/vl.c:3790
i = 64
snapshot = 0
linux_boot = 0
icount_option = 0x0
initrd_filename = 0x0
kernel_filename = 0x0
kernel_cmdline = 0x5555558d032a ""
boot_devices = "dc", '\000' <repeats 30 times>
ds = 0x55555653faf0
dcl = 0x0
cyls = 0
heads = 0
secs = 0
translation = 0
hda_opts = 0x0
opts = 0x55555650a120
machine_opts = 0x55555650a920
olist = 0x5780f65ad2e0
optind = 38
optarg = 0x7fffffffebd8 "cirrus"
loadvm = 0x0
machine = 0x555555c615c0
cpu_model = 0x7fffffffeb5a "host,+x2apic,model_id=Intel(R)
Xeon(R) CPU", ' ' <repeats 11 times>, "L5640 @ 2.27GHz,-tsc"
vga_model = 0x7fffffffebd8 "cirrus"
pid_file = 0x7fffffffeb19 "/var/run/qemu/vm-254.pid"
incoming = 0x0
show_vnc_port = 0
defconfig = true
userconfig = true
log_mask = 0x0
log_file = 0x0
mem_trace = {malloc = 0x5555557231fa <malloc_and_trace>,
realloc = 0x555555723252 <realloc_and_trace>,
free = 0x5555557232b9 <free_and_trace>, calloc = 0,
try_malloc = 0,
try_realloc = 0}
trace_events = 0x0
trace_file = 0x0
(gdb) q
Kevin
Re: [Qemu-devel] Block Migration Assertion in qemu-kvm 1.2.0, Peter Lieven, 2012/09/17