[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Running >10 jobs in parallel on the same remote machine

From: Ole Tange
Subject: Re: Running >10 jobs in parallel on the same remote machine
Date: Sat, 4 Aug 2012 10:17:18 +0200

On Fri, Aug 3, 2012 at 9:25 PM, Thomas Sattler
<> wrote:
> Am 03.08.2012 17:52, schrieb Ole Tange:
>>> How about the "Control" feature of (open-)ssh?
>> Can you test if ControlPersist works for this:
>>     seq 100 | parallel -j0 ssh server 'echo {};sleep 2'
> I have almost no problems. ('parallel' is to fast, and
> the control channel needs some time to be initialized)
> So I get several
> "ControlSocket already exists, disabling multiplexing"

Sounds promising.

> But in case the control channel has been established
> in advance, there are no error messages. So I think
> the following should work:
> let parallel
>  - run one "extra" ssh to the target host
>  - give it time to establish its connections
>  - run all regular ssh's through this
>  - close the "extra" ssh when done
> This should work on all systems that have the "basic"
> ControlMaster feature.

I would like to have people test the idea before I implement it. I
need your help with the options:

  ssh <<what options here?>> server sleep 10000000 &
  sleep 3 # To make sure the above is logged in and set up
  seq 1000 | parallel -j0 ssh <<what options here?>> server 'echo {};sleep 2'
  kill %1 # Kill the sleeping ssh

When we have the options, I will encourage all users to try it out and
report back any problems - I am particularly interested in non-linux
machines, old distributions, slow machines and slow connections, as
that is where I anticipate problems.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]