[PATCH] {linux, bsd}-user: Fail mmap() if size doesn't fit into host's size_t

[Ilya Leoshkevich]

s390x's branch-relative-long test fails with the following error
message on 32-bit hosts:

    qemu-s390x: ../accel/tcg/user-exec.c:493: page_set_flags: Assertion `last 
<= GUEST_ADDR_MAX' failed.

The root cause is that the size passed to mmap() by this test does not
fit into 32 bits and gets truncated. Since there is no chance for such
mmap() to succeed, detect this condition and fail the mmap() right away.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 bsd-user/mmap.c   | 4 ++++
 linux-user/mmap.c | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/bsd-user/mmap.c b/bsd-user/mmap.c
index 3ef11b28079..5dc327d0ad3 100644
--- a/bsd-user/mmap.c
+++ b/bsd-user/mmap.c
@@ -256,6 +256,10 @@ static abi_ulong mmap_find_vma_aligned(abi_ulong start, 
abi_ulong size,
 
     size = HOST_PAGE_ALIGN(size);
 
+    if (size != (size_t)size) {
+        return (abi_ulong)(-1);
+    }
+
     if (reserved_va) {
         return mmap_find_vma_reserved(start, size,
             (alignment != 0 ? 1 << alignment :
diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index 96c9433e271..ae59d70fb67 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -389,6 +389,10 @@ abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size, 
abi_ulong align)
 
     size = HOST_PAGE_ALIGN(size);
 
+    if (size != (size_t)size) {
+        return (abi_ulong)(-1);
+    }
+
     if (reserved_va) {
         return mmap_find_vma_reserved(start, size, align);
     }
-- 
2.43.0