Re: sshloginfile in GNU parallel

[yacob sen]

Dear ole,

Thank you for your reply. I am not really forcing GNU parallel to send specific job to a specific core of on a specific node. I merely showing the 

distribution of the job that GNU parallel supposed to do. I do not care which job goes to which core. I only need that a single task goes to a single core.

I do not understand why you did not specify the "-j" command to show how many cores this

task has to "consume"  on the sample command you have provided in your email. i.e,

  cat  $file_name | parallel --sshlogin node_1_server_address --colsep '  '   program.sh {1} {2}

I expect it to be:

cat  $file_name | parallel -j12 --sshlogin node_1_server_address --colsep '  '   program.sh {1} {2}

or

cat  $file_name | parallel -j+0 --sshlogin node_1_server_address --colsep '  '   program.sh {1} {2}

Regards

Yacob

On Monday, 2 December 2013, 0:54, Ole Tange <ole@tange.dk> wrote:

On Sun, Dec 1, 2013 at 10:31 PM, yacob sen <yacob_123@yahoo.com> wrote:

> Here is my understanding of the gnu parallel implementations on a single
> node:
>
> cat  $file_name | parallel -u -j 12  --sshloginfile  $login_server_name
> --colsep '  '  program.sh {1} {2}

> My idea is for the above command is to distribute each of my jobs to each of
> the cores of a single node. If my implementation is correct, this is what I
> expect the gnu  parallel to do:
>
> At core 1 of node 1:
>
> "program.sh 1    1"  ought to run
>
> At core 2 of node 1:
>
> "program.sh  1    2"  ought to run

You are trying to do something that GNU Parallel is not built for:
Forcing a given program to be run on a specific core on a specific
node. You need to look at 'taskset' to do that. It is unclear why you
need a specific job to be run on a specific core and why another core
is not just as good.

What GNU Parallel excels at is taking a bunch of tasks and run a
number of tasks on one or more computers. So what GNU Parallel _can_
do is to detect the number of cores on the machine and run one job per
cpu core.

In your case it would be something like:

  cat  $file_name | parallel --sshlogin node_1_server_address --colsep

'  '  program.sh {1} {2}

If your arguments really are 1 and 1..12, then you do not need a filename:

  parallel --sshlogin node_1_server_address program.sh {1} {2} ::: 1 ::: {1..12}

If you have not already please spend an hour walking through the
tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html
You will have a much firmer grip of how GNU Parallel works after
walking through it.

The examples in the man-page will also help you understand how to use
GNU Parallel.


/Ole