[Qemu-commits] [qemu/qemu] 6d0785: dirty-bitmap: operate with int64_t amount

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 6d07859926fdb8515fd5cb94a692df7896c3848e
      
https://github.com/qemu/qemu/commit/6d07859926fdb8515fd5cb94a692df7896c3848e
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/dirty-bitmap.c
    M include/block/block_int.h
    M include/block/dirty-bitmap.h

  Log Message:
  -----------
  dirty-bitmap: operate with int64_t amount

Underlying HBitmap operates even with uint64_t. Thus this change is safe.
This would be useful f.e. to mark entire bitmap dirty in one call.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: 531509ba2856ca8efdcf0df18a7dec66af0d36ee
      
https://github.com/qemu/qemu/commit/531509ba2856ca8efdcf0df18a7dec66af0d36ee
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: make sectors_in_flight int64_t

We keep here the sum of int fields. Thus this could easily overflow,
especially when we will start sending big requests in next patches.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: 49efb1f5b0ea45bdb75d578460a7866a58167d1f
      
https://github.com/qemu/qemu/commit/49efb1f5b0ea45bdb75d578460a7866a58167d1f
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: create mirror_throttle helper

The patch also places last_pause_ns from stack in mirror_run into
MirrorBlockJob structure. This helper will be useful in next patches.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
CC: Vladimir Sementsov-Ogievskiy <address@hidden>
CC: Eric Blake <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
CC: Fam Zheng <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: c0b363ad43b00d77c81813db20eec45923973549
      
https://github.com/qemu/qemu/commit/c0b363ad43b00d77c81813db20eec45923973549
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: create mirror_dirty_init helper for mirror_run

The code inside the helper will be extended in the next patch. mirror_run
itself is overbloated at the moment.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy<address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
CC: Eric Blake <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: 2f0342efdbb0a3c5b5a6a4f4831cbe90c445510b
      
https://github.com/qemu/qemu/commit/2f0342efdbb0a3c5b5a6a4f4831cbe90c445510b
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: remove extra condition in bdrv_can_write_zeroes_with_unmap

All .bdrv_co_write_zeroes callbacks nowadays work perfectly even
with backing store attached. If future new callbacks would be unable to do
that - they have a chance to block this in bdrv_get_info().

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: b7d5062c9cf53f21a555843775392dc03364bbff
      
https://github.com/qemu/qemu/commit/b7d5062c9cf53f21a555843775392dc03364bbff
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: optimize dirty bitmap filling in mirror_run a bit

There is no need to scan allocation tables if we have mark_all_dirty flag
set. Just mark it all dirty.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy<address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: c7c2769c0e5769eaad9d968fe2161505b657e02a
      
https://github.com/qemu/qemu/commit/c7c2769c0e5769eaad9d968fe2161505b657e02a
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: efficiently zero out target

With a bdrv_co_write_zeroes method on a target BDS and when this method
is working as indicated by the bdrv_can_write_zeroes_with_unmap(), zeroes
will not be placed into the wire. Thus the target could be very efficiently
zeroed out. This should be done with the largest chunk possible.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy<address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
CC: Eric Blake <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: 4b5004d9fc5b7d8e4447dc81c2f26477c2d590f7
      
https://github.com/qemu/qemu/commit/4b5004d9fc5b7d8e4447dc81c2f26477c2d590f7
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: improve performance of mirroring of empty disk

We should not take into account zero blocks for delay calculations.
They are not read and thus IO throttling is not required. In the
other case VM migration with 16 Tb QCOW2 disk with 4 Gb of data takes
days.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
CC: Jeff Cody <address@hidden>
CC: Eric Blake <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>


  Commit: cf56a3c632d039d00e29dfe8676321d6d349190c
      
https://github.com/qemu/qemu/commit/cf56a3c632d039d00e29dfe8676321d6d349190c
  Author: Denis V. Lunev <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: fix request throttling in drive-mirror

There are 2 deficiencies here:
- mirror_iteration could start several requests inside. Thus we could
  simply have more in_flight requests than MAX_IN_FLIGHT.
- keeping this in mind throttling in mirror_run which is checking
  s->in_flight == MAX_IN_FLIGHT is wrong.

The patch adds the check and throttling into mirror_iteration and fixes
the check in mirror_run() to be sure.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
CC: Jeff Cody <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Max Reitz <address@hidden>
Signed-off-by: Jeff Cody <address@hidden>
(cherry picked from commit e648dc95c28fbca12e67be26a1fc4b9a0676c3fe)


  Commit: d5cf4079ca713cccd16302159fc06f8540c33a57
      
https://github.com/qemu/qemu/commit/d5cf4079ca713cccd16302159fc06f8540c33a57
  Author: Prasanna Kumar Kalever <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/gluster.c

  Log Message:
  -----------
  block/gluster: rename [server, volname, image] -> [host, volume, path]

A future patch will add support for multiple gluster servers. Existing
terminology is a bit unusual in relation to what names are used by
other networked devices, and doesn't map very well to the terminology
we expect to use for multiple servers.  Therefore, rename the following
options:
'server'  -> 'host'
'image'   -> 'path'
'volname' -> 'volume'

Signed-off-by: Prasanna Kumar Kalever <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>
Message-id: address@hidden
Signed-off-by: Jeff Cody <address@hidden>


  Commit: f70c50c81746ecd352617f2a15eca5cb03cf6219
      
https://github.com/qemu/qemu/commit/f70c50c81746ecd352617f2a15eca5cb03cf6219
  Author: Prasanna Kumar Kalever <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/gluster.c

  Log Message:
  -----------
  block/gluster: code cleanup

unified coding styles of multiline function arguments and other error functions
moved random declarations of structures and other list variables

Signed-off-by: Prasanna Kumar Kalever <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>
Message-id: address@hidden
Signed-off-by: Jeff Cody <address@hidden>


  Commit: 0552ff24656c26b2a0d135966fb0feaa90d7f9bc
      
https://github.com/qemu/qemu/commit/0552ff24656c26b2a0d135966fb0feaa90d7f9bc
  Author: Prasanna Kumar Kalever <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/gluster.c

  Log Message:
  -----------
  block/gluster: deprecate rdma support

gluster volfile server fetch happens through unix and/or tcp, it doesn't
support volfile fetch over rdma. The rdma code may actually mislead,
so to make sure things do not break, for now we fallback to tcp when requested
for rdma, with a warning.

If you are wondering how this worked all these days, its the gluster libgfapi
code which handles anything other than unix transport as socket/tcp, sad but
true.

Also gluster doesn't support ipv6 addresses, removing the ipv6 related
comments/docs section

[Jeff: Minor grammatical fixes in comments and commit message, per
review comments]

Signed-off-by: Prasanna Kumar Kalever <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>
Message-id: address@hidden
Signed-off-by: Jeff Cody <address@hidden>


  Commit: 7edac2ddebbc11000ce71244e13b2c9462bf7ad5
      
https://github.com/qemu/qemu/commit/7edac2ddebbc11000ce71244e13b2c9462bf7ad5
  Author: Prasanna Kumar Kalever <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/gluster.c
    M qapi/block-core.json

  Log Message:
  -----------
  block/gluster: using new qapi schema

this patch adds 'GlusterServer' related schema in qapi/block-core.json

[Jeff: minor fix-ups of comments and formatting, per patch reviews]

Signed-off-by: Prasanna Kumar Kalever <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Jeff Cody <address@hidden>


  Commit: 6c7189bb29de9fa2202f613f3c6caf028f96f261
      
https://github.com/qemu/qemu/commit/6c7189bb29de9fa2202f613f3c6caf028f96f261
  Author: Prasanna Kumar Kalever <address@hidden>
  Date:   2016-07-19 (Tue, 19 Jul 2016)

  Changed paths:
    M block/gluster.c
    M qapi/block-core.json

  Log Message:
  -----------
  block/gluster: add support for multiple gluster servers

This patch adds a way to specify multiple volfile servers to the gluster
block backend of QEMU with tcp|rdma transport types and their port numbers.

Problem:

Currently VM Image on gluster volume is specified like this:

file=gluster[+tcp]://host[:port]/testvol/a.img

Say we have three hosts in a trusted pool with replica 3 volume in action.
When the host mentioned in the command above goes down for some reason,
the other two hosts are still available. But there's currently no way
to tell QEMU about them.

Solution:

New way of specifying VM Image on gluster volume with volfile servers:
(We still support old syntax to maintain backward compatibility)

Basic command line syntax looks like:

Pattern I:
 -drive driver=gluster,
  volume=testvol,path=/path/a.raw,[debug=N,]
  server.0.type=tcp,
  server.0.host=1.2.3.4,
  server.0.port=24007,
  server.1.type=unix,
  server.1.socket=/path/socketfile

Pattern II:
 'json:{"driver":"qcow2","file":{"driver":"gluster",
       "volume":"testvol","path":"/path/a.qcow2",["debug":N,]
       "server":[{hostinfo_1}, ...{hostinfo_N}]}}'

   driver      => 'gluster' (protocol name)
   volume      => name of gluster volume where our VM image resides
   path        => absolute path of image in gluster volume
  [debug]      => libgfapi loglevel [(0 - 9) default 4 -> Error]

  {hostinfo}   => {{type:"tcp",host:"1.2.3.4"[,port=24007]},
             {type:"unix",socket:"/path/sockfile"}}

   type        => transport type used to connect to gluster management daemon,
            it can be tcp|unix
   host        => host address (hostname/ipv4/ipv6 addresses/socket path)
   port        => port number on which glusterd is listening.
   socket      => path to socket file

Examples:
1.
 -drive driver=qcow2,file.driver=gluster,
  file.volume=testvol,file.path=/path/a.qcow2,file.debug=9,
  file.server.0.type=tcp,
  file.server.0.host=1.2.3.4,
  file.server.0.port=24007,
  file.server.1.type=unix,
  file.server.1.socket=/var/run/glusterd.socket
2.
  'json:{"driver":"qcow2","file":{"driver":"gluster","volume":"testvol",
   "path":"/path/a.qcow2","debug":9,"server":
   [{"type":"tcp","host":"1.2.3.4","port":"24007"},
    {"type":"unix","socket":"/var/run/glusterd.socket"}
   ]}}'

This patch gives a mechanism to provide all the server addresses, which are in
replica set, so in case host1 is down VM can still boot from any of the
active hosts.

This is equivalent to the backup-volfile-servers option supported by
mount.glusterfs (FUSE way of mounting gluster volume)

credits: sincere thanks to all the supporters

Signed-off-by: Prasanna Kumar Kalever <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Message-id: address@hidden
Signed-off-by: Jeff Cody <address@hidden>


  Commit: e0ce97f896ac752b734c22a6d0d1b3e1739d1158
      
https://github.com/qemu/qemu/commit/e0ce97f896ac752b734c22a6d0d1b3e1739d1158
  Author: Peter Maydell <address@hidden>
  Date:   2016-07-20 (Wed, 20 Jul 2016)

  Changed paths:
    M block.c
    M block/dirty-bitmap.c
    M block/gluster.c
    M block/mirror.c
    M include/block/block_int.h
    M include/block/dirty-bitmap.h
    M qapi/block-core.json

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into 
staging

# gpg: Signature made Wed 20 Jul 2016 01:18:39 BST
# gpg:                using RSA key 0xBDBE7B27C0DE3057
# gpg: Good signature from "Jeffrey Cody <address@hidden>"
# gpg:                 aka "Jeffrey Cody <address@hidden>"
# gpg:                 aka "Jeffrey Cody <address@hidden>"
# Primary key fingerprint: 9957 4B4D 3474 90E7 9D98  D624 BDBE 7B27 C0DE 3057

* remotes/cody/tags/block-pull-request:
  block/gluster: add support for multiple gluster servers
  block/gluster: using new qapi schema
  block/gluster: deprecate rdma support
  block/gluster: code cleanup
  block/gluster: rename [server, volname, image] -> [host, volume, path]
  mirror: fix request throttling in drive-mirror
  mirror: improve performance of mirroring of empty disk
  mirror: efficiently zero out target
  mirror: optimize dirty bitmap filling in mirror_run a bit
  block: remove extra condition in bdrv_can_write_zeroes_with_unmap
  mirror: create mirror_dirty_init helper for mirror_run
  mirror: create mirror_throttle helper
  mirror: make sectors_in_flight int64_t
  dirty-bitmap: operate with int64_t amount

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/3b2e6798ffdc...e0ce97f896ac