Re: [Qemu-devel] [PATCH v3 00/14] tests/vm: serial console autoinstall, misc fixes.

[Philippe Mathieu-Daudé]

Cc'ing the Python team.

On 5/30/19 12:18 PM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <address@hidden> writes:
> 
>> On 5/30/19 11:33 AM, Alex Bennée wrote:
>>>
>>> Gerd Hoffmann <address@hidden> writes:
>>>
>>>> This patch series changes the way virtual machines for test builds are
>>>> managed.  They are created locally on the developer machine now.  The
>>>> installer is booted on the serial console and the scripts walks through
>>>> the dialogs to install and configure the guest.
>>>>
>>>> That takes the download.patchew.org server out of the loop and makes it
>>>> alot easier to tweak the guest images (adding build dependencies for
>>>> example).
>>>>
>>>> The install scripts take care to apply host proxy settings (from *_proxy
>>>> environment variables) to the guest, so any package downloads will be
>>>> routed through the proxy and can be cached that way.  This also makes
>>>> them work behind strict firewalls.
>>>>
>>>> There are also a bunch of smaller tweaks for tests/vm to fix issues I
>>>> was struggling with.  See commit messages of individual patches for
>>>> details.
>>>
>>> Queued to testing/next, thanks.
>>>
>>> One of the machines I'm testing on seems to have problems with getting
>>> the installer working over the serial link but it works on my main dev
>>> box and others have it working as well so I suspect it might be a local
>>> problem.
>>
>> Is this the same issue I described there?
>> https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html
> 
> Not quite.. on the failing machine I see it hang at:
> 
>   make[1]: Leaving directory '/home/alex.bennee/lsrc/qemu.git/dtc'
>   python3 -B /home/alex.bennee/lsrc/qemu.git/tests/vm/openbsd  --debug 
> --image "/home/alex.bennee/.cache/qemu-vm/images/openbsd.img" --force 
> --build-image /home/alex.bennee/.cache/qemu-vm/images/openbsd.img
>   ### Downloading install iso ...
>   ### Preparing iso and disk image ...
>   Formatting '/home/alex.bennee/.cache/qemu-vm/images/openbsd.img.tmp', 
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off 
> refcount_bits=16
>   ### Booting installer ...
>   DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev 
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device virtio-net-pci,netdev=vnet -vnc 
> 127.0.0.1:0,to=20 -smp 18 -enable-kvm
>   -device VGA -drive 
> file=/home/alex.bennee/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
>  -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off -cdrom 
> /home/alex.bennee/.cache/qemu-vm/images/openbsd.img.install.iso
>   DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev 
> socket,id=mon,path=/var/tmp/tmp0uvsee9z/qemu-18506-monitor.sock -mon 
> chardev=mon,mode=control -display none -vga none -machine pc -chardev 
> socket,id=console,path=/var/tmp/tmp0uvsee9z/qemu-18506-console.sock,server,nowait
>  -serial chardev:console -nodefaults -m 4G -cpu max -netdev 
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device virtio-net-pci,netdev=vnet -vnc 
> 127.0.0.1:0,to=20 -smp 18 -enable-kvm -device VGA -drive 
> file=/home/alex.bennee/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
>  -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off -cdrom 
> /home/alex.bennee/.cache/qemu-vm/images/openbsd.img.install.iso'
>   DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
>   DEBUG:QMP:<<< {'return': {}}
>   DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments': 
> {'command-line': 'info usernet'}}
>   DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD  
> Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n  TCP[HOST_FORWARD] 
>  13       127.0.0.1 33465       10.0.2.15    22     0     0\r\n'}
>   console: *** read timeout ***
>   console: waiting for: 'boot>'
>   console: line buffer:
> 
> 
>   Failed to prepare guest environment
>   Traceback (most recent call last):
>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/basevm.py", line 350, in 
> main
>       return vm.build_image(args.image)
>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/openbsd", line 85, in 
> build_image
>       self.console_wait_send("boot>", "set tty com0\n")
>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/basevm.py", line 250, in 
> console_wait_send
>       self.console_wait(wait)
>     File "/home/alex.bennee/lsrc/qemu.git/tests/vm/basevm.py", line 212, in 
> console_wait
>       chars = vm.console_socket.recv(1024)
>   socket.timeout: timed out
>   DEBUG:QMP:>>> {'execute': 'quit'}
>   DEBUG:QMP:<<< {'return': {}}
>   /home/alex.bennee/lsrc/qemu.git/tests/vm/Makefile.include:47: recipe for 
> target '/home/alex.bennee/.cache/qemu-vm/images/openbsd.img' failed
>   make: *** [/home/alex.bennee/.cache/qemu-vm/images/openbsd.img] Error 2
> 
> I can see the machine has actually booted as VNC can connect but for
> some reason the serial isn't working.

Looks related IMO :) Some desync with the serial. Maybe some Python
detail we are missing (like an Exception we are silently ignoring).

BTW I can reproduce my case, and it looks you can reproduce yours too.