[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 3/6] util/oslib-posix: Don't create too many threads with
From: |
Dr. David Alan Gilbert |
Subject: |
Re: [PATCH v2 3/6] util/oslib-posix: Don't create too many threads with small memory or little pages |
Date: |
Tue, 27 Jul 2021 20:01:15 +0100 |
User-agent: |
Mutt/2.0.7 (2021-05-04) |
* David Hildenbrand (david@redhat.com) wrote:
> Let's limit the number of threads to something sane, especially that
> - We don't have more threads than the number of pages we have
> - We don't have threads that initialize small (< 64 MiB) memory
>
> Signed-off-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> util/oslib-posix.c | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/util/oslib-posix.c b/util/oslib-posix.c
> index 2ae6c3aaab..a1d309d495 100644
> --- a/util/oslib-posix.c
> +++ b/util/oslib-posix.c
> @@ -40,6 +40,7 @@
> #include <libgen.h>
> #include "qemu/cutils.h"
> #include "qemu/compiler.h"
> +#include "qemu/units.h"
>
> #ifdef CONFIG_LINUX
> #include <sys/syscall.h>
> @@ -529,7 +530,8 @@ static void *do_madv_populate_write_pages(void *arg)
> return NULL;
> }
>
> -static inline int get_memset_num_threads(int smp_cpus)
> +static inline int get_memset_num_threads(size_t hpagesize, size_t numpages,
> + int smp_cpus)
> {
> long host_procs = sysconf(_SC_NPROCESSORS_ONLN);
> int ret = 1;
> @@ -537,6 +539,12 @@ static inline int get_memset_num_threads(int smp_cpus)
> if (host_procs > 0) {
> ret = MIN(MIN(host_procs, MAX_MEM_PREALLOC_THREAD_COUNT), smp_cpus);
> }
> +
> + /* Especially with gigantic pages, don't create more threads than pages.
> */
> + ret = MIN(ret, numpages);
> + /* Don't start threads to prealloc comparatively little memory. */
> + ret = MIN(ret, MAX(1, hpagesize * numpages / (64 * MiB)));
> +
> /* In case sysconf() fails, we fall back to single threaded */
> return ret;
> }
> @@ -546,7 +554,7 @@ static bool touch_all_pages(char *area, size_t hpagesize,
> size_t numpages,
> {
> static gsize initialized = 0;
> MemsetContext context = {
> - .num_threads = get_memset_num_threads(smp_cpus),
> + .num_threads = get_memset_num_threads(hpagesize, numpages, smp_cpus),
> };
> size_t numpages_per_thread, leftover;
> void *(*touch_fn)(void *);
> --
> 2.31.1
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
[PATCH v2 2/6] util/oslib-posix: Introduce and use MemsetContext for touch_all_pages(), David Hildenbrand, 2021/07/22
[PATCH v2 3/6] util/oslib-posix: Don't create too many threads with small memory or little pages, David Hildenbrand, 2021/07/22
[PATCH v2 4/6] util/oslib-posix: Avoid creating a single thread with MADV_POPULATE_WRITE, David Hildenbrand, 2021/07/22
[PATCH v2 5/6] util/oslib-posix: Support concurrent os_mem_prealloc() invocation, David Hildenbrand, 2021/07/22
[PATCH v2 6/6] util/oslib-posix: Forward SIGBUS to MCE handler under Linux, David Hildenbrand, 2021/07/22