[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL v3 32/55] util/oslib-posix: Don't create too many threads with sma
From: |
Michael S. Tsirkin |
Subject: |
[PULL v3 32/55] util/oslib-posix: Don't create too many threads with small memory or little pages |
Date: |
Fri, 7 Jan 2022 20:05:15 -0500 |
From: David Hildenbrand <david@redhat.com>
Let's limit the number of threads to something sane, especially that
- We don't have more threads than the number of pages we have
- We don't have threads that initialize small (< 64 MiB) memory
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20211217134611.31172-5-david@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
util/oslib-posix.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index cf2ead54ad..67c08a425e 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -40,6 +40,7 @@
#include <libgen.h>
#include "qemu/cutils.h"
#include "qemu/compiler.h"
+#include "qemu/units.h"
#ifdef CONFIG_LINUX
#include <sys/syscall.h>
@@ -525,7 +526,8 @@ static void *do_madv_populate_write_pages(void *arg)
return (void *)(uintptr_t)ret;
}
-static inline int get_memset_num_threads(int smp_cpus)
+static inline int get_memset_num_threads(size_t hpagesize, size_t numpages,
+ int smp_cpus)
{
long host_procs = sysconf(_SC_NPROCESSORS_ONLN);
int ret = 1;
@@ -533,6 +535,12 @@ static inline int get_memset_num_threads(int smp_cpus)
if (host_procs > 0) {
ret = MIN(MIN(host_procs, MAX_MEM_PREALLOC_THREAD_COUNT), smp_cpus);
}
+
+ /* Especially with gigantic pages, don't create more threads than pages. */
+ ret = MIN(ret, numpages);
+ /* Don't start threads to prealloc comparatively little memory. */
+ ret = MIN(ret, MAX(1, hpagesize * numpages / (64 * MiB)));
+
/* In case sysconf() fails, we fall back to single threaded */
return ret;
}
@@ -542,7 +550,7 @@ static int touch_all_pages(char *area, size_t hpagesize,
size_t numpages,
{
static gsize initialized = 0;
MemsetContext context = {
- .num_threads = get_memset_num_threads(smp_cpus),
+ .num_threads = get_memset_num_threads(hpagesize, numpages, smp_cpus),
};
size_t numpages_per_thread, leftover;
void *(*touch_fn)(void *);
--
MST
- [PULL v3 22/55] vhost-user-blk: propagate error return from generic vhost, (continued)
- [PULL v3 22/55] vhost-user-blk: propagate error return from generic vhost, Michael S. Tsirkin, 2022/01/07
- [PULL v3 21/55] vhost: stick to -errno error return convention, Michael S. Tsirkin, 2022/01/07
- [PULL v3 24/55] pcie_aer: Don't trigger a LSI if none are defined, Michael S. Tsirkin, 2022/01/07
- [PULL v3 23/55] pci: Export the pci_intx() function, Michael S. Tsirkin, 2022/01/07
- [PULL v3 26/55] hw/smbios: Use qapi for SmbiosEntryPointType, Michael S. Tsirkin, 2022/01/07
- [PULL v3 25/55] smbios: Rename SMBIOS_ENTRY_POINT_* enums, Michael S. Tsirkin, 2022/01/07
- [PULL v3 27/55] hw/i386: expose a "smbios-entry-point-type" PC machine property, Michael S. Tsirkin, 2022/01/07
- [PULL v3 31/55] util/oslib-posix: Introduce and use MemsetContext for touch_all_pages(), Michael S. Tsirkin, 2022/01/07
- [PULL v3 28/55] hw/vhost-user-blk: turn on VIRTIO_BLK_F_SIZE_MAX feature for virtio blk device, Michael S. Tsirkin, 2022/01/07
- [PULL v3 29/55] util/oslib-posix: Let touch_all_pages() return an error, Michael S. Tsirkin, 2022/01/07
- [PULL v3 32/55] util/oslib-posix: Don't create too many threads with small memory or little pages,
Michael S. Tsirkin <=
- [PULL v3 33/55] util/oslib-posix: Avoid creating a single thread with MADV_POPULATE_WRITE, Michael S. Tsirkin, 2022/01/07
- [PULL v3 30/55] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc(), Michael S. Tsirkin, 2022/01/07
- [PULL v3 36/55] virtio-mem: Support "prealloc=on" option, Michael S. Tsirkin, 2022/01/07
- [PULL v3 34/55] util/oslib-posix: Support concurrent os_mem_prealloc() invocation, Michael S. Tsirkin, 2022/01/07
- [PULL v3 35/55] util/oslib-posix: Forward SIGBUS to MCE handler under Linux, Michael S. Tsirkin, 2022/01/07
- [PULL v3 37/55] virtio: signal after wrapping packed used_idx, Michael S. Tsirkin, 2022/01/07
- [PULL v3 38/55] MAINTAINERS: Add a separate entry for acpi/VIOT tables, Michael S. Tsirkin, 2022/01/07
- [PULL v3 39/55] linux-headers: sync VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE, Michael S. Tsirkin, 2022/01/07
- [PULL v3 40/55] virtio-mem: Support VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE, Michael S. Tsirkin, 2022/01/07
- [PULL v3 41/55] virtio-mem: Set "unplugged-inaccessible=auto" for the 7.0 machine on x86, Michael S. Tsirkin, 2022/01/07