Re: [Qemu-devel] Memory use with >100 virtio devices

[David Gibson]

On Fri, Aug 18, 2017 at 02:18:53PM +0100, Stefan Hajnoczi wrote:
> On Fri, Aug 18, 2017 at 03:39:20PM +1000, Alexey Kardashevskiy wrote:
> > ==94451== -------------------- 4 of 10 --------------------
> > ==94451== max-live:    314,649,600 in 150 blocks
> > ==94451== tot-alloc:   314,649,600 in 150 blocks (avg size 2097664.00)
> > ==94451== deaths:      none (none of these blocks were freed)
> > ==94451== acc-ratios:  0.00 rd, 0.00 wr  (0 b-read, 0 b-written)
> > ==94451==    at 0x4895600: memalign (in
> > /usr/lib/valgrind/vgpreload_exp-dhat-ppc64le-linux.so)
> > ==94451==    by 0x48957E7: posix_memalign (in
> > /usr/lib/valgrind/vgpreload_exp-dhat-ppc64le-linux.so)
> > ==94451==    by 0xB744AB: qemu_try_memalign (oslib-posix.c:106)
> > ==94451==    by 0xA92053: qemu_try_blockalign (io.c:2493)
> > ==94451==    by 0xA34DDF: qcow2_do_open (qcow2.c:1365)
> > ==94451==    by 0xA35627: qcow2_open (qcow2.c:1526)
> > ==94451==    by 0x9FB94F: bdrv_open_driver (block.c:1109)
> > ==94451==    by 0x9FC413: bdrv_open_common (block.c:1365)
> > ==94451==    by 0x9FF823: bdrv_open_inherit (block.c:2542)
> > ==94451==    by 0x9FFC17: bdrv_open (block.c:2626)
> > ==94451==    by 0xA71027: blk_new_open (block-backend.c:267)
> > ==94451==    by 0x6D3E6B: blockdev_init (blockdev.c:588)
> 
> This allocation is unnecessary.  Most qcow2 files are not encrypted so
> s->cluster_data does not need to be allocated upfront.
> 
> I'll send a patch.

Is that sufficient to explain the problem, I can't quickly see how big
that unnecessary allocation is - but would it account for the 10s of
gigabytes usage we're seeing here?

I'm suspecting we accidentally have a O(n^2) or worse space complexity
going on here.

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature