[Qemu-commits] [qemu/qemu] bb2e16: scripts/device-crash-test: Remove devices that are...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: bb2e16e90bec9fbdd7bd86dfa2e6ca1c894bc3d8
      
https://github.com/qemu/qemu/commit/bb2e16e90bec9fbdd7bd86dfa2e6ca1c894bc3d8
  Author: Thomas Huth <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M scripts/device-crash-test

  Log Message:
  -----------
  scripts/device-crash-test: Remove devices that are not user_creatable anymore

Devices that are derived from TYPE_SYS_BUS_DEVICE are not user_creatable
anymore by default, and some others have been marked as non-user_creatable
manually, so we can remove these devices from the "ignore"-list in the
device-crash-test script.

Signed-off-by: Thomas Huth <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 6d39db96d23ebe0d4361d108fa202091aa9cbfc1
      
https://github.com/qemu/qemu/commit/6d39db96d23ebe0d4361d108fa202091aa9cbfc1
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/qemu-iotests/083.out
    M tests/qemu-iotests/nbd-fault-injector.py

  Log Message:
  -----------
  iotests: Make nbd-fault-injector flush

When closing a connection, make the nbd-fault-injector flush the socket.
Without this, the output is a bit unreliable with Python 3.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: f544adf736cb1b100a7fe926f3712157b22c3ddd
      
https://github.com/qemu/qemu/commit/f544adf736cb1b100a7fe926f3712157b22c3ddd
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: Flush in iotests.py's QemuIoInteractive

After issuing a command, flush the pipe.  This does not change anything
in Python 2, but it makes a difference in Python 3.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 8eb5e6746feaf9e021b69ea2521899f8dc889033
      
https://github.com/qemu/qemu/commit/8eb5e6746feaf9e021b69ea2521899f8dc889033
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M scripts/qtest.py
    M tests/qemu-iotests/044
    M tests/qemu-iotests/149
    M tests/qemu-iotests/207
    M tests/qemu-iotests/iotests.py
    M tests/qemu-iotests/nbd-fault-injector.py
    M tests/qemu-iotests/qcow2.py

  Log Message:
  -----------
  iotests: Use Python byte strings where appropriate

Since byte strings are no longer the default in Python 3, we have to
explicitly use them where we need to, which is mostly when working with
structures.  It also means that we need to open a file in binary mode
when we want to use structures.

On the other hand, we have to accomodate for the fact that some
functions (still) work with byte strings but we want to use unicode
strings (in Python 3 at least, and it does not matter in Python 2).
This includes base64 encoding, but it is most notable when working with
the subprocess module: Either we set universal_newlines to True so that
the default streams are opened in text mode (hence this parameter is
aliased as "text" as of 3.7), or, if that is not possible, we have to
decode the output to a normal string.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 9a3a9a636eaf207816891f504b569b8d674987aa
      
https://github.com/qemu/qemu/commit/9a3a9a636eaf207816891f504b569b8d674987aa
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/qemu-iotests/030
    M tests/qemu-iotests/040
    M tests/qemu-iotests/041
    M tests/qemu-iotests/044
    M tests/qemu-iotests/093
    M tests/qemu-iotests/136
    M tests/qemu-iotests/149
    M tests/qemu-iotests/151
    M tests/qemu-iotests/163
    M tests/qemu-iotests/iotests.py
    M tests/qemu-iotests/qed.py

  Log Message:
  -----------
  iotests: Use // for Python integer division

In Python 3, / is always a floating-point division.  We usually do not
want this, and as Python 2.7 understands // as well, change all integer
divisions to use that.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 68474776f354bc728db58cf349fe3ed0f63a306e
      
https://github.com/qemu/qemu/commit/68474776f354bc728db58cf349fe3ed0f63a306e
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/qemu-iotests/044
    M tests/qemu-iotests/056
    M tests/qemu-iotests/065
    M tests/qemu-iotests/124
    M tests/qemu-iotests/139
    M tests/qemu-iotests/163

  Log Message:
  -----------
  iotests: Different iterator behavior in Python 3

In Python 3, several functions now return iterators instead of lists.
This includes range(), items(), map(), and filter().  This means that if
we really want a list, we have to wrap those instances with list().  But
then again, the two instances where this is the case for map() and
filter(), there are shorter expressions which work without either
function.

On the other hand, sometimes we do just want an iterator, in which case
we have sometimes used xrange() and iteritems() which no longer exist in
Python 3.  Just change these calls to be range() and items(), works in
both Python 2 and 3, and is really what we want in 3 (which is what
matters).  But because it is so simple to do (and to find and remove
once we completely switch to Python 3), make range() be an alias for
xrange() in the two affected tests (044 and 163).

In one instance, we only wanted the first instance of the result of a
filter() call.  Instead of using next(filter()) which would work only in
Python 3, or list(filter())[0] which would work everywhere but is a bit
weird, this instance is changed to use a generator expression with a
next() wrapped around, which works both in 2.7 and 3.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: bf43b29df4cdc716f4a2af98b37e206a11541d99
      
https://github.com/qemu/qemu/commit/bf43b29df4cdc716f4a2af98b37e206a11541d99
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M scripts/qemu.py
    M tests/qemu-iotests/045
    M tests/qemu-iotests/147

  Log Message:
  -----------
  iotests: Explicitly bequeath FDs in Python

Python 3.4 introduced the inheritable attribute for FDs.  At the same
time, it changed the default so that all FDs are not inheritable by
default, that only inheritable FDs are inherited to subprocesses, and
only if close_fds is explicitly set to False.

Adhere to this by setting close_fds to False when working with
subprocesses that may want to inherit FDs, and by trying to
set_inheritable() on FDs that we do want to bequeath to them.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: c1a65cba9c1b9a0ad72046dc73ab0e316b9cb7fe
      
https://github.com/qemu/qemu/commit/c1a65cba9c1b9a0ad72046dc73ab0e316b9cb7fe
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/qemu-iotests/169

  Log Message:
  -----------
  iotests: 'new' module replacement in 169

iotest 169 uses the 'new' module to add methods to a class.  This module
no longer exists in Python 3.  Instead, we can use a lambda.  Best of
all, this works in 2.7 just as well.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 2d894bee1c907399fbd59163a18e42c41daf4863
      
https://github.com/qemu/qemu/commit/2d894bee1c907399fbd59163a18e42c41daf4863
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/qemu-iotests/iotests.py
    M tests/qemu-iotests/nbd-fault-injector.py

  Log Message:
  -----------
  iotests: Modify imports for Python 3

There are two imports that need to be modified when running the iotests
under Python 3: One is StringIO, which no longer exists; instead, the
StringIO class comes from the io module, so import it from there (and
use the BytesIO class for Python 2).  The other is the ConfigParser,
which has just been renamed to configparser.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: e21b5f34d669b82087597273f3783626947291a0
      
https://github.com/qemu/qemu/commit/e21b5f34d669b82087597273f3783626947291a0
  Author: Max Reitz <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/qemu-iotests/194.out
    M tests/qemu-iotests/202.out
    M tests/qemu-iotests/203.out
    M tests/qemu-iotests/206.out
    M tests/qemu-iotests/207
    M tests/qemu-iotests/207.out
    M tests/qemu-iotests/208.out
    M tests/qemu-iotests/210.out
    M tests/qemu-iotests/211.out
    M tests/qemu-iotests/212.out
    M tests/qemu-iotests/213.out
    M tests/qemu-iotests/216.out
    M tests/qemu-iotests/218.out
    M tests/qemu-iotests/219.out
    M tests/qemu-iotests/222.out
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: Unify log outputs between Python 2 and 3

When dumping an object into the log, there are differences between
Python 2 and 3.  First, unicode strings are prefixed by 'u' in Python 2
(they are no longer in 3, because unicode strings are the default
there).  Second, the order of keys in dicts may differ.  Third,
especially long numbers are longs in Python 2 and thus get an 'L'
suffix, which does not happen in Python 3.

We can get around all of these differences by dumping objects (lists and
dicts) in a language-independent format, namely JSON.  The JSON
generator even allows emitting dicts with their keys sorted
alphabetically.

This changes the output of all tests that use these logging functions
(dict keys are ordered now, strings in dicts are now enclosed in double
quotes instead of single quotes, the 'L' suffix of large integers is
dropped, and "true" and "false" are now in lower case).
The quote change necessitates a small change to a filter used in test
207.

Suggested-by: Eduardo Habkost <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 213137217a60eca18e9b55817f00dfdd6eaff74a
      
https://github.com/qemu/qemu/commit/213137217a60eca18e9b55817f00dfdd6eaff74a
  Author: Cleber Rosa <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M tests/Makefile.include
    A tests/requirements.txt

  Log Message:
  -----------
  Bootstrap Python venv for tests

A number of QEMU tests are written in Python, and may benefit
from an untainted Python venv.

By using make rules, tests that depend on specific Python libs
can set that rule as a requirement, along with rules that require
the presence or installation of specific libraries.

The tests/requirements.txt is supposed to contain the Python
requirements that should be added to the venv created by check-venv.

Signed-off-by: Cleber Rosa <address@hidden>
Tested-by: Philippe Mathieu-Daudé <address@hidden>
Acked-by: Stefan Hajnoczi <address@hidden>
Acked-by: Wainer dos Santos Moschetta <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: a56931eef343c7564e35bcc05eaed2a469a1b1b8
      
https://github.com/qemu/qemu/commit/a56931eef343c7564e35bcc05eaed2a469a1b1b8
  Author: Cleber Rosa <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M docs/devel/testing.rst
    M tests/Makefile.include
    M tests/requirements.txt

  Log Message:
  -----------
  Acceptance tests: add make rule for running them

The acceptance (aka functional, aka Avocado-based) tests are
Python files located in "tests/acceptance" that need to be run
with the Avocado libs and test runner.

Let's provide a convenient way for QEMU developers to run them,
by making use of the tests-venv with the required setup.

Also, while the Avocado test runner will take care of creating a
location to save test results to, it was understood that it's better
if the results are kept within the build tree.

Signed-off-by: Cleber Rosa <address@hidden>
Acked-by: Stefan Hajnoczi <address@hidden>
Acked-by: Wainer dos Santos Moschetta <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: aa983ff67c365a91bdf819c89654d4e7afd5943d
      
https://github.com/qemu/qemu/commit/aa983ff67c365a91bdf819c89654d4e7afd5943d
  Author: Cleber Rosa <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M .travis.yml

  Log Message:
  -----------
  Travis support for the acceptance tests

This enables the execution of the acceptance tests on Travis.

Because the Travis environment is based on Ubuntu Trusty, it requires
the python3-pip and python3.4-venv packages.

Signed-off-by: Cleber Rosa <address@hidden>
Tested-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Acked-by: Stefan Hajnoczi <address@hidden>
Acked-by: Wainer dos Santos Moschetta <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: cbcdf1a951aa605c43dc80a98a8cc366299e378c
      
https://github.com/qemu/qemu/commit/cbcdf1a951aa605c43dc80a98a8cc366299e378c
  Author: Cleber Rosa <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M scripts/decodetree.py

  Log Message:
  -----------
  scripts/decodetree.py: fix reference to attributes

Signed-off-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: e301e65c97142c4d2a2adf35f5a4fa73d65d8a4f
      
https://github.com/qemu/qemu/commit/e301e65c97142c4d2a2adf35f5a4fa73d65d8a4f
  Author: Cleber Rosa <address@hidden>
  Date:   2018-10-30 (Tue, 30 Oct 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  scripts/qemu.py: use a more consistent docstring style

Signed-off-by: Cleber Rosa <address@hidden>
Message-Id: <address@hidden>
[ehabkost: reverted unintentional submodule update]
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 8ebf8ea90ecbbb2ef743dccfbfb73e2904cad1f5
      
https://github.com/qemu/qemu/commit/8ebf8ea90ecbbb2ef743dccfbfb73e2904cad1f5
  Author: Peter Maydell <address@hidden>
  Date:   2018-11-01 (Thu, 01 Nov 2018)

  Changed paths:
    M .travis.yml
    M docs/devel/testing.rst
    M scripts/decodetree.py
    M scripts/device-crash-test
    M scripts/qemu.py
    M scripts/qtest.py
    M tests/Makefile.include
    M tests/qemu-iotests/030
    M tests/qemu-iotests/040
    M tests/qemu-iotests/041
    M tests/qemu-iotests/044
    M tests/qemu-iotests/045
    M tests/qemu-iotests/056
    M tests/qemu-iotests/065
    M tests/qemu-iotests/083.out
    M tests/qemu-iotests/093
    M tests/qemu-iotests/124
    M tests/qemu-iotests/136
    M tests/qemu-iotests/139
    M tests/qemu-iotests/147
    M tests/qemu-iotests/149
    M tests/qemu-iotests/151
    M tests/qemu-iotests/163
    M tests/qemu-iotests/169
    M tests/qemu-iotests/194.out
    M tests/qemu-iotests/202.out
    M tests/qemu-iotests/203.out
    M tests/qemu-iotests/206.out
    M tests/qemu-iotests/207
    M tests/qemu-iotests/207.out
    M tests/qemu-iotests/208.out
    M tests/qemu-iotests/210.out
    M tests/qemu-iotests/211.out
    M tests/qemu-iotests/212.out
    M tests/qemu-iotests/213.out
    M tests/qemu-iotests/216.out
    M tests/qemu-iotests/218.out
    M tests/qemu-iotests/219.out
    M tests/qemu-iotests/222.out
    M tests/qemu-iotests/iotests.py
    M tests/qemu-iotests/nbd-fault-injector.py
    M tests/qemu-iotests/qcow2.py
    M tests/qemu-iotests/qed.py
    A tests/requirements.txt

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' 
into staging

Python queue, 2018-10-30

* Makefile rule for running acceptance tests
  (make check-acceptance) (Cleber Rosa)
* Make iotests compatible with Python 3
  (Max Reitz)
* device-crash-test whitelist update (Thomas Huth)
* Misc cleanups (Cleber Rosa)

# gpg: Signature made Wed 31 Oct 2018 00:28:39 GMT
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <address@hidden>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
  scripts/qemu.py: use a more consistent docstring style
  scripts/decodetree.py: fix reference to attributes
  Travis support for the acceptance tests
  Acceptance tests: add make rule for running them
  Bootstrap Python venv for tests
  iotests: Unify log outputs between Python 2 and 3
  iotests: Modify imports for Python 3
  iotests: 'new' module replacement in 169
  iotests: Explicitly bequeath FDs in Python
  iotests: Different iterator behavior in Python 3
  iotests: Use // for Python integer division
  iotests: Use Python byte strings where appropriate
  iotests: Flush in iotests.py's QemuIoInteractive
  iotests: Make nbd-fault-injector flush
  scripts/device-crash-test: Remove devices that are not user_creatable anymore

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/8002fa2bf6d3...8ebf8ea90ecb
      **NOTE:** This service has been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.