[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8'
From: |
Eduardo Habkost |
Subject: |
Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8' |
Date: |
Mon, 18 Jun 2018 18:35:26 -0300 |
User-agent: |
Mutt/1.9.2 (2017-12-15) |
On Mon, Jun 18, 2018 at 07:59:57PM +0200, Markus Armbruster wrote:
> Python 2 happily reads UTF-8 files in text mode, but Python 3 requires
> either UTF-8 locale or an explicit encoding passed to open(). Commit
> d4e5ec877ca fixed this by setting the en_US.UTF-8 locale. Falls apart
> when the locale isn't be available.
>
> Matthias Maier and Arfrever Frehtes Taifersar Arahesis proposed to use
> binary mode instead, with manual conversion from bytes to str. Works,
> but opening with an explicit encoding is simpler, so do that.
>
> Since Python 2's open() doesn't support the encoding parameter, we
> need to suppress it with a version check.
>
> Reported-by: Arfrever Frehtes Taifersar Arahesis <address@hidden>
> Reported-by: Matthias Maier <address@hidden>
> Signed-off-by: Markus Armbruster <address@hidden>
> ---
> scripts/qapi/common.py | 17 ++++++++++++++---
> 1 file changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index 2462fc0291..832f11438a 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -16,6 +16,7 @@ import errno
> import os
> import re
> import string
> +import sys
> from collections import OrderedDict
>
> builtin_types = {
> @@ -340,7 +341,10 @@ class QAPISchemaParser(object):
> return None
>
> try:
> - fobj = open(incl_fname, 'r')
> + if sys.version_info[0] >= 3:
> + fobj = open(incl_fname, 'r', encoding='utf-8')
> + else:
> + fobj = open(incl_fname, 'r')
I dislike the Python version check, but getting rid of it would
require rewriting the QAPI modules to not use the Python 2 str
type (that has different semantics from Python 3 str type).
The python-future package would help us write code for a single
file/string API instead of two different APIs, but it's not a
QEMU build dependency (yet?), so this patch is good enough for
now.
Reviewed-by: Eduardo Habkost <address@hidden>
Acked-by: Eduardo Habkost <address@hidden>
> except IOError as e:
> raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname))
> return QAPISchemaParser(fobj, previously_included, info)
> @@ -1492,7 +1496,11 @@ class QAPISchemaEvent(QAPISchemaEntity):
> class QAPISchema(object):
> def __init__(self, fname):
> self._fname = fname
> - parser = QAPISchemaParser(open(fname, 'r'))
> + if sys.version_info[0] >= 3:
> + f = open(fname, 'r', encoding='utf-8')
> + else:
> + f = open(fname, 'r')
> + parser = QAPISchemaParser(f)
> exprs = check_exprs(parser.exprs)
> self.docs = parser.docs
> self._entity_list = []
> @@ -2006,7 +2014,10 @@ class QAPIGen(object):
> if e.errno != errno.EEXIST:
> raise
> fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666)
> - f = os.fdopen(fd, 'r+')
> + if sys.version_info[0] >= 3:
> + f = open(fd, 'r+', encoding='utf-8')
> + else:
> + f = os.fdopen(fd, 'r+')
> text = (self._top(fname) + self._preamble + self._body
> + self._bottom(fname))
> oldtext = f.read(len(text) + 1)
> --
> 2.17.1
>
>
--
Eduardo
- [Qemu-devel] [PATCH v4 0/2] Fix compilation with python-3 if en_US.UTF-8 is unavailable, Markus Armbruster, 2018/06/18
- [Qemu-devel] [PATCH v4 2/2] Revert commit d4e5ec877ca, Markus Armbruster, 2018/06/18
- [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8', Markus Armbruster, 2018/06/18
- Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8',
Eduardo Habkost <=
- Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8', Markus Armbruster, 2018/06/19
- Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8', Eduardo Habkost, 2018/06/19
- Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8', Markus Armbruster, 2018/06/19
- Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8', Daniel P . Berrangé, 2018/06/19
- Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8', Eduardo Habkost, 2018/06/19
Re: [Qemu-devel] [PATCH v4 1/2] qapi: Open files with encoding='utf-8', Eric Blake, 2018/06/19