Re: [PULL 0/2] block: Fix iotests to respect configured Python binary

[Kevin Wolf]

Am 29.01.2021 um 17:13 hat Peter Maydell geschrieben:
> On Fri, 29 Jan 2021 at 14:52, Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > The following changes since commit 5101d00d2f1138a73344dc4833587f76d7a5fa5c:
> >
> >   Merge remote-tracking branch 
> > 'remotes/vivier2/tags/trivial-branch-for-6.0-pull-request' into staging 
> > (2021-01-29 10:10:43 +0000)
> >
> > are available in the Git repository at:
> >
> >   git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 4cea90be62f4f15a63e1a8f7d5d0958f79fdf290:
> >
> >   tests/Makefile.include: export PYTHON for check-block.sh (2021-01-29 
> > 12:32:36 +0100)
> >
> > ----------------------------------------------------------------
> > Block layer patches:
> >
> > - Fix iotests to respect configured Python binary
> >
> > ----------------------------------------------------------------
> 
> This is definitely better so I'm going to apply it, but it seems
> to reveal a pile of iotest failures on FreeBSD:
> [...]
> Failures: 030 040 041 127 256

Hm... I did run 'make check-block' on a FreeBSD VM before sending the
pull request, but it turns out I ran it on the wrong branch. *sigh*

After rerunning it with the right one (and manually so that I get all
tests, not just those in 'make check-block'), however, while I'm seeing
some failures, my list of failing cases and your list are completely
disjoint:

    Failures: 115 125 145 182 286 298 300 migrate-bitmaps-postcopy-test
    migrate-bitmaps-test

All of these are test cases that are not run during 'make check-block',
so I would assume that they were already broken before.

So there seems to be something more specific to the environment that
made your test cases fail than just FreeBSD.

> though the entire build goes on to succeed (which is probably
> a test framework bug somewhere...)

This is not good, and it can easily be reproduced on Linux by simply
changing the reference output of a test so that it fails. 'check'
doesn't seem to return the right exit code any more.

Kevin