Re: GNU Parallel as Job queue Manager

[Ole Tange]

On Mon, May 20, 2013 at 8:22 PM, Vivek Raghunathan <nkrvivek@gmail.com> wrote:
> Hi Ole,

Please use parallel@gnu.org for support (unless you want to hire me).

> First of all thank you for your contribution to the open source
> community.

Thank you. If you like GNU Parallel:

* Give a demo at your local user group/team/colleagues
* Post the intro videos on Reddit/Diaspora*/forums/blogs/
Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
* Request or write a review for your favourite blog or magazine
* Invite me for your next conference

If you use GNU Parallel for research:

* Please cite GNU Parallel in you publications (use --bibtex)

If GNU Parallel saves you money:

* (Have your company) donate to FSF https://my.fsf.org/donate/

> I'm trying to get the job queue manager working on Ubuntu
> 10.04.2 LTS.

>From 'man parallel':

       Your bug report should always include:

       · The error message you get (if any).

       · The output of parallel --version. If you are not running
         the latest released version you should specify why you
         believe the problem is not fixed in that version.

       · A complete example that others can run that shows the
         problem. This should preferably be small and simple. A
         combination of yes, seq, cat, echo, and sleep can
         reproduce most errors. If your example requires large
         files, see if you can make them by something like seq
         1000000 > file or yes | head -n 10000000 > file.

       · The output of your example. If your problem is not
         easily reproduced by others, the output might help them
         figure out the problem.

       If you suspect the error is dependent on your environment
       or distribution, please see if you can reproduce the error
       on one of these VirtualBox images:
       http://sourceforge.net/projects/virtualboximage/files/

       Specifying the name of your distribution is not enough as
       you may have installed software that is not in the
       VirtualBox images.

       If you cannot reproduce the error on any of the VirtualBox
       images above, you should assume the debugging will be done
       through you. That will put more burden on you and it is
       extra important you give any information that help.

> I'm using these commands to achieve that, but even though I'm
> submitting jobs by echoing commands to jobqueue, It's not getting
> executed properly for some reason.
>
> Looks like the tail -f is not passing the commands to parallel
> properly or parallel is not picking it up like a job manager. FYI, I'm
> using parallel_20120422-1_all.deb
>
> echo >jobqueue; tail -f jobqueue | parallel
>
> To submit your jobs to the queue:
> echo my_command my_arg >> jobqueue
>
> Am I doing something wrong?

There is nothing in what you do, that seems to be wrong. But there may
be something locally on your machine that prevents it from working, so
please try it on a VirtualBox. And please include a complete working
example. If the problem also exists on the VirtualBox I will be able
to reproduce your problems.

> Any help would be much appreciated.
>
> Vivek

/Ole