Re: parallel + blast + LSF

[Ole Tange]

On Mon, May 4, 2015 at 5:31 PM, Giuseppe Aprea <giuseppe.aprea@gmail.com> wrote:

> it looks like all jobs are executed on the first server regardless of server
> file. I am using:
>
> cat proteins.fasta | parallel --no-notice -vv -j ${SLOTS} --tmpdir tmp
> --wait --slf servers --block 200k --recstart '>' --pipe blastp -evalue 1e-05
> -outfmt 6 -db proteins -query - -out result_{#}
>
> (SLOTS=192)
>
> with serverfile given by:
>
>> more servers
> 24/cresco3x045.portici.enea.it

The 24 tells GNU Parallel to treat this machine as having 24 cores.
This number is used in the -j calculation. -j 100% would mean 24 jobs
for this machine, -j 50% would mean 12 jobs, -j -1 would mean 23. And
-j 192 would mean 192.

So you have probably misunderstood how -j works. My advice: Remove -j
and let GNU Parallel run one job per core.

> It also
> looks like option "--wait" is neglected and GNU parallel is trying to run
> all 192 instances at the same time on a single server.

In the manual it says:

      --wait   Wait for all commands to complete.

                Implies --semaphore.

                See also man sem.

Did you read about --semaphore? Did you try out --wait in the
tutorial? Is that really what you want?

Maybe GNU Parallel should fail when using --pipe with
--wait/--semaphore? I cannot see a situation in which the combination
will make sense.


/Ole

PS: Consider running --bibtex once.