parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: parallel + blast + LSF


From: Giuseppe Aprea
Subject: Re: parallel + blast + LSF
Date: Mon, 4 May 2015 17:31:00 +0200

Hi, 

it looks like all jobs are executed on the first server regardless of server file. I am using:

cat proteins.fasta | parallel --no-notice -vv -j ${SLOTS} --tmpdir tmp --wait --slf servers --block 200k --recstart '>' --pipe blastp -evalue 1e-05 -outfmt 6 -db proteins -query - -out result_{#}

(SLOTS=192)

with serverfile given by:

> more servers 
24/cresco3x045.portici.enea.it
16/cresco3x047.portici.enea.it
16/cresco3x049.portici.enea.it
24/cresco3x060.portici.enea.it
24/cresco3x065.portici.enea.it
24/cresco3x073.portici.enea.it
16/cresco3x078.portici.enea.it
24/cresco3x079.portici.enea.it
24/cresco3x080.portici.enea.it

for i in `cat servers |sed -e s:^[0-9]*/::g` ; do echo "------------------------------------------------- "$i ; ssh $i ps -ef |egrep blastp |wc -l ; done
------------------------------------------------- cresco3x045.portici.enea.it
0
------------------------------------------------- cresco3x047.portici.enea.it
0
------------------------------------------------- cresco3x049.portici.enea.it
0
------------------------------------------------- cresco3x060.portici.enea.it
0
------------------------------------------------- cresco3x065.portici.enea.it
614
------------------------------------------------- cresco3x073.portici.enea.it
0
------------------------------------------------- cresco3x078.portici.enea.it
0
------------------------------------------------- cresco3x079.portici.enea.it
0
------------------------------------------------- cresco3x080.portici.enea.it
0



According to manual (https://www.gnu.org/software/parallel/man.html) I expected  a maximum of 24 blastp instances on cresco3x045.portici.enea.it, a maximum of 16 blastp instances on cresco3x047.portici.enea.it, etc. It also looks like option "--wait" is neglected and GNU parallel is trying to run all 192 instances at the same time on a single server.

Could anyone tell me, please, what's going wrong?

Thanks in advance,

Giuseppe






On Thu, Apr 23, 2015 at 1:56 PM, Giuseppe Aprea <address@hidden> wrote:
Hi!

Mea culpa, I am sorry. You are right, I should have tried the ncpu/sshlogin syntax before writing my message.

g





On Thu, Apr 23, 2015 at 1:32 AM, Ole Tange <address@hidden> wrote:
On Wed, Apr 22, 2015 at 9:57 AM, Giuseppe Aprea
<address@hidden> wrote:

> Anyway now my main concern is how do I specify the maximum job number per
> host when I am using multiple worker nodes?

Have you read the section on --sshlogin and --slf in the man page?


/Ole




reply via email to

[Prev in Thread] Current Thread [Next in Thread]