[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Properly scheduling multi-threaded/process jobs

From: Douglas A. Augusto
Subject: Re: Properly scheduling multi-threaded/process jobs
Date: Sun, 31 Aug 2014 13:19:25 -0300
User-agent: Mutt/1.5.23 (2014-03-12)

On 31/08/2014 at 09:59,
Ole Tange <> wrote:

> So while I am happy to learn that you appreciate GNU Parallel's
> features, then instead of expanding GNU Parallel into a full queuing
> system I would prefer seeing an extension that would make it possible
> for GNU Parallel to interface to real queuing systems with proper
> resource management (such as Torque). That way we can re-use
> development already done instead of re-inventing the wheel.

Hi, I couldn't agree more. I don't know the internals of GNU Parallel's
scheduling kernel, but perhaps an intermediate approach would be to make it
easy to plug in user-defined schedulers by means of an interface.

Regarding the handling of multi-process jobs within GNU Parallel, for now I'm
using a work-around: instead of running the job directly I use a wrapper that
only starts the job if there are enough resources to run it; otherwise it will
sleep for awhile (of course there are some side-effects, such as making
inconsistent the use of --timeout).

Douglas A. Augusto

reply via email to

[Prev in Thread] Current Thread [Next in Thread]