Re: [Qemu-devel] [PULL 19/19] configure: automatically pick python3 is available

[Eduardo Habkost]

On Fri, May 03, 2019 at 06:00:11PM -0300, Eduardo Habkost wrote:
> On Fri, May 03, 2019 at 06:41:43PM +0200, Thomas Huth wrote:
> > On 03/05/2019 02.41, Eduardo Habkost wrote:
> > > From: Daniel P. Berrangé <address@hidden>
> > > 
> > > Unless overridden via an env var or configure arg, QEMU will only look
> > > for the 'python' binary in $PATH. This is unhelpful on distros which
> > > are only shipping Python 3.x (eg Fedora) in their default install as,
> > > if they comply with PEP 394, the bare 'python' binary won't exist.
> > > 
> > > This changes configure so that by default it will search for all three
> > > common python binaries, preferring to find Python 3.x versions.
> > > 
> > > Signed-off-by: Daniel P. Berrangé <address@hidden>
> > > Message-Id: <address@hidden>
> > > Signed-off-by: Eduardo Habkost <address@hidden>
> > > ---
> > >  configure | 18 +++++++++++++++---
> > >  1 file changed, 15 insertions(+), 3 deletions(-)
> > 
> > I haven't bisected it, but I think this patch here broke the gitlab-ci 
> > tests:
> > 
> >  https://gitlab.com/huth/qemu/-/jobs/206806257
> > 
> > Seems like the test is now failing when you don't have an UTF-8 locale:
> > 
> >  LANG=C make check-qapi-schema
> 
> I couldn't reproduce it this way, probably because I'm running Python 3.7 
> which
> implements PEP 538 ("Coercing the legacy C locale to a UTF-8 based locale").
> 
> But I can force it to break using:
> 
>   PYTHONIOENCODING=ascii make check-qapi-schema
> 
> >  [...]
> >  TEST    tests/qapi-schema/union-base-empty.out
> >  --- /builds/huth/qemu/tests/qapi-schema/unicode-str.err    2019-05-03 
> > 15:21:39.000000000 +0000
> >  +++ -      2019-05-03 15:42:01.561762978 +0000
> >  @@ -1 +1 @@
> >  -tests/qapi-schema/unicode-str.json:2: 'command' uses invalid name 'é'
> >  +tests/qapi-schema/unicode-str.json:2: 'command' uses invalid name '\xe9'
> >  /builds/huth/qemu/tests/Makefile.include:1105: recipe for target 
> > 'check-tests/qapi-schema/unicode-str.json' failed
> >  make: *** [check-tests/qapi-schema/unicode-str.json] Error 1
> > 
> > Any ideas how to fix this?
> 
> Probably we just need to specify an explicit encoding at the statement that
> prints the error message to stderr.  I will give it a try.

Forcing a specific encoding inside test-qapi.py would very easy
on Python 3.7+ (sys.stderr.reconfigure(...)), but tricky on older
versions.  I believe this is the simplest way to fix the problem
on Python 3.5 and 3.6.

Can somebody confirm this really fixes the problem on gitlab-ci?

---
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 7c8b9c84b2..af88ab6f8b 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -1103,7 +1103,7 @@ check-tests/qemu-iotests-quick.sh: 
tests/qemu-iotests-quick.sh qemu-img$(EXESUF)
 .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
 $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: 
$(SRC_PATH)/%.json
        $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts \
-               $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \
+               PYTHONIOENCODING=utf-8 $(PYTHON) 
$(SRC_PATH)/tests/qapi-schema/test-qapi.py \
                $^ >$*.test.out 2>$*.test.err; \
                echo $$? >$*.test.exit, \
                "TEST","$*.out")

-- 
Eduardo