Re: [Qemu-devel] [PATCH] linux-user: do setrlimit selectively

[Peter Maydell]

On 4 September 2018 at 22:00, Max Filippov <address@hidden> wrote:
> When running 32-bit guest on 64-bit host setrlimit guest calls that
> affect memory resources (RLIMIT_{AS,DATA,STACK}) don't always make sense
> as is. They may result in QEMU lockup because mprotect call in
> page_unprotect would fail with ENOMEM error code, causing infinite loop
> of SIGSEGV. E.g. it happens when running libstdc++ testsuite for xtensa
> target on x86_64 host.
>
> Don't call host setrlimit for memory-related resources when running
> 32-bit guest on 64-bit host.

I think the issue here is not related to 32-on-64 but to the fact
that we just pass through the memory rlimits. What we should ideally
be doing is tracking the actual guest memory allocations sufficiently
that we can then apply the rlimits at the QEMU level, so that guest
allocations that breach limits can be failed, without ever causing
QEMU's own alloactions to fail. (There's a bug in launchpad somewhere
about this -- an autoconf test for some obscure printf bug tries
to set a low rlimit and makes QEMU fall over.)

Unfortunately adding tracking of how much memory the guest has
allocated is probably non-trivial...

thanks
-- PMM