Re: [Qemu-devel] [PATCH v3 00/14] tests/vm: serial console autoinstall, misc fixes.

[Philippe Mathieu-Daudé]

Cc'ing Paolo & Marc-André (and removing the BSD folks).

On 5/30/19 12:30 PM, Philippe Mathieu-Daudé wrote:
> Cc'ing the Python team.
> 
> On 5/30/19 12:18 PM, Alex Bennée wrote:
>>
>> Philippe Mathieu-Daudé <address@hidden> writes:
>>
>>> On 5/30/19 11:33 AM, Alex Bennée wrote:
>>>>
>>>> Gerd Hoffmann <address@hidden> writes:
>>>>
>>>>> This patch series changes the way virtual machines for test builds are
>>>>> managed.  They are created locally on the developer machine now.  The
>>>>> installer is booted on the serial console and the scripts walks through
>>>>> the dialogs to install and configure the guest.
>>>>>
>>>>> That takes the download.patchew.org server out of the loop and makes it
>>>>> alot easier to tweak the guest images (adding build dependencies for
>>>>> example).
>>>>>
>>>>> The install scripts take care to apply host proxy settings (from *_proxy
>>>>> environment variables) to the guest, so any package downloads will be
>>>>> routed through the proxy and can be cached that way.  This also makes
>>>>> them work behind strict firewalls.
>>>>>
>>>>> There are also a bunch of smaller tweaks for tests/vm to fix issues I
>>>>> was struggling with.  See commit messages of individual patches for
>>>>> details.
>>>>
>>>> Queued to testing/next, thanks.
>>>>
>>>> One of the machines I'm testing on seems to have problems with getting
>>>> the installer working over the serial link but it works on my main dev
>>>> box and others have it working as well so I suspect it might be a local
>>>> problem.
>>>
>>> Is this the same issue I described there?
>>> https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html
>>
>> Not quite.. on the failing machine I see it hang at:
>>
>>   make[1]: Leaving directory '/home/alex.bennee/lsrc/qemu.git/dtc'
>>   python3 -B /home/alex.bennee/lsrc/qemu.git/tests/vm/openbsd  --debug 
>> --image "/home/alex.bennee/.cache/qemu-vm/images/openbsd.img" --force 
>> --build-image /home/alex.bennee/.cache/qemu-vm/images/openbsd.img
>>   ### Downloading install iso ...
>>   ### Preparing iso and disk image ...
>>   Formatting '/home/alex.bennee/.cache/qemu-vm/images/openbsd.img.tmp', 
>> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off 
>> refcount_bits=16
>>   ### Booting installer ...
>>   DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev 
>> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device virtio-net-pci,netdev=vnet 
>> -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
>>   -device VGA -drive 
>> file=/home/alex.bennee/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
>>  -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off -cdrom 
>> /home/alex.bennee/.cache/qemu-vm/images/openbsd.img.install.iso
>>   DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev 
>> socket,id=mon,path=/var/tmp/tmp0uvsee9z/qemu-18506-monitor.sock -mon 
>> chardev=mon,mode=control -display none -vga none -machine pc -chardev 
>> socket,id=console,path=/var/tmp/tmp0uvsee9z/qemu-18506-console.sock,server,nowait
>>  -serial chardev:console -nodefaults -m 4G -cpu max -netdev 
>> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device virtio-net-pci,netdev=vnet 
>> -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm -device VGA -drive 
>> file=/home/alex.bennee/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
>>  -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off -cdrom 
>> /home/alex.bennee/.cache/qemu-vm/images/openbsd.img.install.iso'
>>   DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
>>   DEBUG:QMP:<<< {'return': {}}
>>   DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments': 
>> {'command-line': 'info usernet'}}
>>   DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD  
>> Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n  
>> TCP[HOST_FORWARD]  13       127.0.0.1 33465       10.0.2.15    22     0     
>> 0\r\n'}
>>   console: *** read timeout ***
>>   console: waiting for: 'boot>'
>>   console: line buffer:
>>
>>
>>   Failed to prepare guest environment
>>   Traceback (most recent call last):
>>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/basevm.py", line 350, in 
>> main
>>       return vm.build_image(args.image)
>>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/openbsd", line 85, in 
>> build_image
>>       self.console_wait_send("boot>", "set tty com0\n")
>>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/basevm.py", line 250, in 
>> console_wait_send
>>       self.console_wait(wait)
>>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/basevm.py", line 212, in 
>> console_wait
>>       chars = vm.console_socket.recv(1024)
>>   socket.timeout: timed out
>>   DEBUG:QMP:>>> {'execute': 'quit'}
>>   DEBUG:QMP:<<< {'return': {}}
>>   /home/alex.bennee/lsrc/qemu.git/tests/vm/Makefile.include:47: recipe for 
>> target '/home/alex.bennee/.cache/qemu-vm/images/openbsd.img' failed
>>   make: *** [/home/alex.bennee/.cache/qemu-vm/images/openbsd.img] Error 2
>>
>> I can see the machine has actually booted as VNC can connect but for
>> some reason the serial isn't working.
> 
> Looks related IMO :) Some desync with the serial. Maybe some Python
> detail we are missing (like an Exception we are silently ignoring).
> 
> BTW I can reproduce my case, and it looks you can reproduce yours too.

I now see a similar issue with Avocado.
Using this patch
https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg05747.html
I get:

$ make check-venv
$ ./tests/venv/bin/python -m avocado --show=console run -t arch:rx
tests/acceptance/boot_linux_console.py
console: Linux version 4.19.0+ (address@hidden) (gcc version
9.0.0 20181105 (experimental) (GCC)) #137 Wed Feb 20 23:20:02 JST 2019
console: Built 1 zonelists, mobility grouping on.  Total pages: 8128
console: Kernel command line:
console: Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
console: Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
console: Memory: 14648K/32768K available (871K kernel code, 95K rwdata,
140K rodata, 96K init, 175K bss, 18120K reserved, 0K cma-reserved)
console: NR_IRQS: 256
console: rx-cmt: used for periodic clock events
console: clocksource: rx-tpu: mask: 0xffffffff max_cycles: 0xffffffff,
max_idle_ns: 1274173631191 ns
console: 96.00 BogoMIPS (lpj=480000)
console: pid_max: default: 4096 minimum: 301
console: Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
console: Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
console: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff,
max_idle_ns: 19112604462750000 ns
console: clocksource: Switched to clocksource rx-tpu
console: workingset: timestamp_bits=30 max_order=12 bucket_order=0
console: SuperH (H)SCI(F) driver initialized
console: 88240.serial: ttySC0 at MMIO 0x88240 (irq = 215, base_baud = 0)
is a sci
console: console [ttySC0] enabled
console: 88248.serial: ttySC1 at MMIO 0x88248 (irq = 219, base_baud = 0)
is a sci
console: random: get_random_bytes called from 0x01002e48 with crng_init=0
console: Freeing unused kernel memory: 96K
console: This architecture does not have kernel memory protection.
console: Run /sbin/init as init process
console: Run /etc/init as init process
console: Run /bin/init as init process
console: Run /bin/sh as init process
console:
console: Sash command shell (version 1.1.1)
console: />sh-sci 88240.serial: overrun error
console: sh-sci 88240.serial: frame error
console: sh-sci 88240.serial: parity error
console: prrandom: fast init done
[HANG]

Most of the time this test succeed.

console: This architecture does not have kernel memory protection.
console: Run /sbin/init as init process
console: Run /etc/init as init process
console: Run /bin/init as init process
console: Run /bin/sh as init process
console:
console: Sash command shell (version 1.1.1)
console: /> printenv
console: HOME=/
console: TERM=linux
$ echo $?
0