[PATCH] Linux-user: umap redundant reserved memory in loading ELF

[Xie Benyi]

When loading ELF, linux-user reserves address space, from the lowest
address of all segments in the ELF, to the highest address of all
segments in the ELF. If the segments in some ELF file (for example,
wine) scatter among the whole guest address space, then Linux-user
will reserve almost the whole guest memory. As a result, the guest
application has nearly no memory, which causing mmap system failed.
So unmaps redundant memory which are not needed by the ELF segments.

Signed-off-by: Xie Benyi <xieby1@outlook.com>
---
 linux-user/elfload.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 04c28cb..6c5d449 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2494,6 +2494,26 @@ static void load_elf_image(const char *image_name, int 
image_fd,
         goto exit_perror;
     }
     load_bias = load_addr - loaddr;
+    /*
+     * Unmap the memory which not needed by ELF segments.
+     *
+     * Instead of directly reserving the needed memory space,
+     * we first target_mmap all space betwenn loaddr and hiaddr,
+     * then target_munmap the holes between space needed by segments.
+     * It is because directly reserving the needed memory will call
+     * target_mmap several times, which may result in different
+     * load_bias for different segments.
+     */
+    abi_ulong munmap_start = -1, munmap_end = 0;
+    for (i = 0; i < ehdr->e_phnum; ++i) {
+        if (phdr[i].p_type == PT_LOAD) {
+            munmap_end = phdr[i].p_vaddr - phdr[i].p_offset;
+            if (munmap_start != -1 && munmap_end > munmap_start) {
+                target_munmap(munmap_start, munmap_end - munmap_start);
+            }
+            munmap_start = phdr[i].p_vaddr + phdr[i].p_memsz;
+        }
+    }
 
     if (elf_is_fdpic(ehdr)) {
         struct elf32_fdpic_loadseg *loadsegs = info->loadsegs =
-- 
2.25.1