Re: Problem with thousands of small jobs.

[Ole Tange]

On Fri, Feb 6, 2015 at 2:20 PM, xmoon 2000 <xmoon2000@googlemail.com> wrote:

> On my 32 core machine works OK,   BUT there is a "lull" in processing
> every few seconds as new jobs are started, once the current crop have
> completed.

I have assumed that the disk is inactive and that GNU Parallel is
taking 100% of a core, and that you would like to use all your cores
to 100%. But maybe that assumption is wrong. Maybe GNU Parallel is
waiting for disk I/O? Maybe the system running low on RAM and swapping
(dataprocessing jobs often use the most memory just before they exit)?

Can you describe this "lull" further? Can you reproduce this "lull" by
running jobs that any Cygwin user can reproduce?


/Ole