bug-parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Warning: Starting 10 extra processes takes > 2 sec.


From: Aaron Paetznick
Subject: Warning: Starting 10 extra processes takes > 2 sec.
Date: Thu, 14 Oct 2010 08:52:59 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100922 Thunderbird/3.1.4

I keep getting the following warning when using parallel:

Warning: Starting 10 extra processes takes > 2 sec.
Limiting to 1 jobs in parallel.


This is on an NFS server that is busy compressing tar archives all day. I appreciate that parallel is trying to protect me from myself, but I feel like I should be able to force the number of processes and/or bypass this warning if I need to.

This happens if I specify anything for -j, including -j2, -j8, and -j+0. If I don't specify -j, it defaults to 9 processes and will correctly run all 9. The irony here is that 9 processes will stress this machine too much, but if I specify say -j4 it will throw a warning and only start one process which will also kill the machine because it can't keep up with the incoming queue of files and then the disks get full.

I've tried messing with nice values and assigning parallel a better scheduler priority, but that's still not 100% effective and a little awkward. Can I request a new feature to force the number of processes (i.e. --force) or at least bypass this check (i.e. --dont-blame-parallel)? Or maybe parallel should only do this check if it's dynamically setting the number of processes (i.e. -j+0, -j50%, etc.) and not if explicitly passed a number (i.e. -j4, etc.).

Thanks.


--Aaron




reply via email to

[Prev in Thread] Current Thread [Next in Thread]