Dear Ole,I am bioinformatician from faculty hospital Motol in Czech Republic.
I am using gnu parallel to speed up NGS computations. I am able to speed up the computations ~4 times in comparison with parallelization implemented within GATK tools:)
In order to fully automatize the ngs pipeline I used --memfree option.. But I went to the problems.
I parallelize every program within my pipeline separately and I use version 20160422 of parallel. In the case of base recalibrator parallelization the computations crashes without any ERROR info. I call parallel as follows:
$pathToParallel --no-notice --joblog "/home/geryk/NGS_programs/GnuParallel/log" --jobs 0 --memfree $inputSize"m" baseRacalibrator_pass1 \
::: "address@hidden" ::: "$pathToProject" ::: "$deactivateFilters"
where $inputSize is my estimate of memory requirements. If I add parameter --delay 15, the computations finishes succesfully.
But if I decrease the parameter to --delay 10 computations crashes.
In the second application of base recalibration (this program must be applied twice within pipeline) I must increase --delay to 30sec in order to successfully finish computations.
It seems that the parameter --delay is hard to predict..
Have you some interpretations of my observations?
Log files from parallel are attached to this e-mail..
Sincerelly,
Jan Geryk
-- Upozornění: Není-li v této zprávě výslovně uvedeno jinak, má
tato e-mailová zpráva nebo její přílohy pouze informativní charakter. Tato
zpráva ani její přílohy v žádném ohledu FN Motol k ničemu nezavazují. Text této
zprávy nebo jejích příloh není návrhem na uzavření smlouvy, ani přijetím
případného návrhu na uzavření smlouvy, ani jiným právním jednáním směřujícím k
uzavření jakékoliv smlouvy a nezakládá předsmluvní odpovědnost FN Motol.
Disclaimer:
If not expressly stated otherwise, this e-mail message (including any attached files) is intended purely for informatory purposes and does not represent a binding agreement on the part of Motol University Hospital. The text of this message and its attachments cannot be considered as a proposal to conclude a contract, nor the acceptance of a proposal to conclude a contract, nor any other legal act leading to concluding any contract; nor does it create any pre-contractual liability on the part of Motol University Hospital.
logs.zip