[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Running GNU parallel with PBS

From: Erik Petigura
Subject: Running GNU parallel with PBS
Date: Wed, 20 Jun 2012 10:43:53 -0700


This is my first post, so let me start off by saying thanks for a great tool!  I am using GNU parallel to manage some data-parallel tasks on a cluster where the resources are managed by PBS.  

Here are my two questions:

1) How do I transfer my environment variables to a remote machine using the --slf option
2) How does parallel read in the config file.  I can't seem to set `--slf $PBS_NODEFILE' in the config file.



My first issue is that my local environment variables are not getting transfered to remote nodes

local> echo $PYTHONPATH /global/home/users/petigura/Kepler//keppycode//
local> seq 0 15 | parallel -j8 --slf $PBS_NODEFILE 'echo {}; echo $PYTHON_PATH; echo $PWD'



I can work around this by explicitly sourcing my bashrc file

local> seq 0 15 | parallel -j8 --slf $PBS_NODEFILE 'source ~/.bashrc;echo {}; echo $PYTHONPATH; echo $PWD'


I'd like to use a config file to simplify my commands, but I can't get it to work.

local> cat ~/.parallel/nodes
#Config file.  Options to parallel when running on multiple nodes
source $HOME/.bashrc;

local > seq 0 15 | parallel -j8 -Jnodes  --wd $PWD 'source ~/.bashrc;echo {}; echo $PYTHONPATH; echo $PWD'
Cannot open $PBS_NODEFILE


Thanks for your help!


reply via email to

[Prev in Thread] Current Thread [Next in Thread]