Advice on using parallel as a batch queue system

[Tong, Lianheng]

Dear all,

This is my first post and I am very pleased to have discovered gnu parallel 
about a month ago. It had saved me a lot hassle while running a lot of serial 
simulation jobs. So a big thanks to the developers :-)

I have however encountered some problems when trying to use parallel as a batch 
queue, and I am not sure if it is a feature of the code or me not doing things 
incorrectly, so any advice would be welcome.  The version of parallel I am 
using is 20121222

As the first try, I did the following:

true > queue
tail -f queue | parallel -j 4 &

echo "echo 1" >> queue

(nothing happened)

echo "touch foo" >> queue

(nothing happened, and ls shows that foo was not created)

echo "echo 2" >> queue

(nothing happened, foo not created)

echo "echo 3" >> queue

(1 is output in stdout, and file foo has been created.)
 
echo "echo 4" >> queue

(nothing happened)

echo "echo 5" >> queue

(2 is displayed in stdout)

echo "touch foo2" >> queue

(3 is displayed in stdout, and file foo2 is created.)

Killing parallel:  kill %1 gives the following output:

parallel: SIGTERM received. No new jobs will be started.
parallel: Waiting for these 3 jobs to finish. Send SIGTERM again to stop now.
petrof: lt$ parallel: touch foo2
parallel: echo 5
parallel: echo 4
4
5

[1]+  Done                    tail -f ../queue | parallel -j 4

So it seems that jobs are only run AFTER job-slots number (in this case 4) of 
jobs have been submitted. If I have only sent 3 jobs to the queue, then none of 
the jobs are executed until I send the SIGTERM to parallel to terminate the 
queue.

On the online documentation, it is said that jobs sent to the queue should run 
immediately (provided there are free slots).  Is this something that has been 
changed in the newer revisions?  Or have I done something incorrect?

Many thanks,

Lianheng

==========================================================
Lianheng Tong                                                             Tel: 
+44 20 7679 3302
London Centre For Nanotechnology                       Fax: +44 20 7679 0595
University College London                                        
lianheng.tong@ucl.ac.uk
17–19 Gordon Street, London WC1H 0AH, U.K.
==========================================================