Re: Cannot specify the number of threads for parsort

[Rob Sargent]

On 2/17/23 13:41, Mario Roy wrote:

It looks like we may not get what we kindly asked for. So, I started making "mcesort" using Perl MCE's chunking engine.

On Thu, Feb 16, 2023 at 5:08 AM Nigel Stewart <nigels@nigels.com> wrote:

Can you elaborate on what I am missing from the picture?

Ole,

Perhaps your workloads are more CPU and I/O intensive, and latency is less of a priority.
If the workload is memory-intensive, that can be the more important constraint than
the number of available cores.  If the workload is interactive (latency-sensitive) it's
undesirable to have too many jobs in flight competing for CPU and I/O, delaying each other.

- Nigel

 

    Are you in the high
      memory consumption scenario which Nigel describes?
      
      If you're going to develop it anyway, you could try submitting a
      patch to GNU Parallel.