[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to
|
From: |
Dr. David Alan Gilbert |
|
Subject: |
Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB |
|
Date: |
Tue, 28 Jun 2016 15:20:07 +0100 |
|
User-agent: |
Mutt/1.6.1 (2016-04-27) |
* Peter Lieven (address@hidden) wrote:
> Am 28.06.2016 um 13:35 schrieb Dr. David Alan Gilbert:
> > * Peter Lieven (address@hidden) wrote:
> > > Am 28.06.2016 um 12:57 schrieb Dr. David Alan Gilbert:
> > > > * Paolo Bonzini (address@hidden) wrote:
> > > > > On 28/06/2016 11:01, Peter Lieven wrote:
> > > > > > evaluation with the recently introduced maximum stack size
> > > > > > monitoring revealed
> > > > > > that the actual used stack size was never above 4kB so allocating
> > > > > > 1MB stack
> > > > > > for each coroutine is a lot of wasted memory. So reduce the stack
> > > > > > size to
> > > > > > 64kB which should still give enough head room.
> > > > > If we make the stack this much smaller, there is a non-zero chance of
> > > > > smashing it. You must add a guard page if you do this (actually more
> > > > > than one because QEMU will happily have stack frames as big as 16 KB).
> > > > > The stack counts for RSS but it's not actually allocated memory, so
> > > > > why
> > > > > does it matter?
> > > > I think I'd be interested in seeing the /proc/.../smaps before and
> > > > after this
> > > > change to see if anything is visible and if we can see the difference
> > > > in rss etc.
> > > Can you advise what in smaps should be especially looked at.
> > >
> > > As for RSS I can report hat the long term usage is significantly lower.
> > > I had the strange observation that when the VM is running for some minutes
> > > the RSS suddenly increases to the whole stack size.
> > You can see the Rss of each mapping; if you knew where your stacks were
> > it would be easy to see if it was the stacks that were Rss and if
> > there was anything else odd about them.
> > If you set hte mapping as growsdown then you can see the area that has a
> > 'gd'
> > in it's VmFlags.
>
> Would you expect to see each 1MB allocation in smaps or is it possible that
> the kernel merges some mappings to bigger ones?
>
> And more importantly if the regions are merged Paolos comment about we
> do not need a guard page would not be true because a coroutine stack could
> grow into annother coroutines stack. Looking at the commit from Linus it
> would also be good to have that guard page not having the gd flag.
Hmm I'm not sure; one for Paolo.
> Some of the regions above 1024kB have an RSS of exactly 4kB * (Size / 1024kB)
> which leads to the assumption that it is a corouine stack where exactly one
> page
> has been allocated.
>
> I am asking because this is what I e.g. see for a Qemu VM with flags "gd":
However, what that does show is that if you add up all the Rss, it's still
near-enough nothing worth worrying about.
Maybe it looks different in the old world before you mmap'd it, you could
try going back to the g_malloc'd version but printf'ing the
address you get, then comparing that with smaps to see what the malloc'd
world ended up with mapped.
Dave
> cat /proc/5031/smaps | grep -B18 gd
> 7f808aee7000-7f808b9e6000 rw-p 00000000 00:00 0
> Size: 11264 kB
> Rss: 44 kB
> Pss: 44 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 44 kB
> Referenced: 44 kB
> Anonymous: 44 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f808bb01000-7f8090000000 rw-p 00000000 00:00 0
> Size: 70656 kB
> Rss: 276 kB
> Pss: 276 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 276 kB
> Referenced: 276 kB
> Anonymous: 276 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f80940ff000-7f80943fe000 rw-p 00000000 00:00 0
> Size: 3072 kB
> Rss: 12 kB
> Pss: 12 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 12 kB
> Referenced: 12 kB
> Anonymous: 12 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f8095700000-7f80957ff000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f8097301000-7f8097400000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f80974df000-7f80975de000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> 7f809760c000-7f809770b000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f8097901000-7f8097a00000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f8097b01000-7f8097c00000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f8097d01000-7f8097e00000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f8197f01000-7f8198000000 rw-p 00000000 00:00 0
> Size: 1024 kB
> Rss: 4 kB
> Pss: 4 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 4 kB
> Referenced: 4 kB
> Anonymous: 4 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7f81b4001000-7f81b4200000 rw-p 00000000 00:00 0
> Size: 2048 kB
> Rss: 20 kB
> Pss: 20 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 20 kB
> Referenced: 20 kB
> Anonymous: 20 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac sd
> --
> 7ffd337e2000-7ffd33805000 rw-p 00000000 00:00 0
> [stack]
> Size: 144 kB
> Rss: 64 kB
> Pss: 64 kB
> Shared_Clean: 0 kB
> Shared_Dirty: 0 kB
> Private_Clean: 0 kB
> Private_Dirty: 64 kB
> Referenced: 64 kB
> Anonymous: 64 kB
> AnonHugePages: 0 kB
> Shared_Hugetlb: 0 kB
> Private_Hugetlb: 0 kB
> Swap: 0 kB
> SwapPss: 0 kB
> KernelPageSize: 4 kB
> MMUPageSize: 4 kB
> Locked: 0 kB
> VmFlags: rd wr mr mw me gd ac
>
> Peter
>
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK
- [Qemu-devel] [PATCH 01/15] coroutine-ucontext: mmap stack memory, (continued)
- [Qemu-devel] [PATCH 01/15] coroutine-ucontext: mmap stack memory, Peter Lieven, 2016/06/28
- [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Peter Lieven, 2016/06/28
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Paolo Bonzini, 2016/06/28
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Dr. David Alan Gilbert, 2016/06/28
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Peter Lieven, 2016/06/28
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Dr. David Alan Gilbert, 2016/06/28
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Peter Lieven, 2016/06/28
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB,
Dr. David Alan Gilbert <=
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Peter Lieven, 2016/06/30
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Peter Lieven, 2016/06/28
- Re: [Qemu-devel] [PATCH 03/15] coroutine-ucontext: reduce stack size to 64kB, Paolo Bonzini, 2016/06/28
[Qemu-devel] [PATCH 02/15] coroutine-ucontext: add a switch to monitor maximum stack size, Peter Lieven, 2016/06/28
[Qemu-devel] [PATCH 15/15] vnc: use mmap for VncState, Peter Lieven, 2016/06/28
[Qemu-devel] [PATCH 14/15] vnc-tight: make the encoding palette static, Peter Lieven, 2016/06/28
[Qemu-devel] [PATCH 13/15] exec: use mmap for PhysPageMap->nodes, Peter Lieven, 2016/06/28