RE: Enhancement/feature idea for GNU parallel

[David Bennion]

I tried the --progress option yesterday.  With lame anyway stuff gets
buffered up and then blasts all at once to the console.  

I still think the separate console feature would be cool -- real time visual
feedback for jobs that potentially run hours and then parallel has its own
command console.

But it's all fine.  I'm happy to have a good concurrent execution utility.
I was able to substantially reduce the time on some stuff with it.

I'm a capable engineer, so if this idea keeps buzzing in my head enough to
get me to take action I'll go implement for myself and send you an optional
patch.  Otherwise, we can let it go.

Cheers.

-----Original Message-----
From: ole.tange@gmail.com [mailto:ole.tange@gmail.com] On Behalf Of Ole
Tange
Sent: Sunday, February 20, 2011 4:57 AM
To: David Bennion
Cc: parallel@gnu.org
Subject: Re: Enhancement/feature idea for GNU parallel

On Sat, Feb 19, 2011 at 5:16 PM, David Bennion <david.bennion@gmx.com>
wrote:
>
> Hey Ole,
>
> I wanted to solve a problem (parallel encoding mp3s and also parallel runs
of ffmpeg).  I didn’t know parallel existed and I had not used xargs before,
but I have been able to really use parallel and gain performance from my
4-way system.  So thanks for your efforts.

Glad you like it.

> The idea that I had that I thought would be neat is if you could pass in N
number of device consoles (/dev/pts/1 /dev/pts/2 /dev/pts/3) and if parallel
would reopen stdout, stderr and stdin for child processes.  The net effect
would be that you could create multiple login shells/windows on your box and
you could watch in real time as each job runs – whichever console it ends up
on.
>
> This is not really something that could be done by the jobs themselves
because the consoles need to be selected based on which processor the jobs
get run on.  It would be an error if not enough console devices were passed
in, they need to match 1 to 1 with processors/job threads.

As I understand you would need to:

1: Open terminals for each process you want run in parallel
2: Run "tty" in each of them to get the devicename
3: Pass these outputs to GNU Parallel on the command line
4: Run the parallel command

It seems like an awful lot of work just to be able to monitor
progress. Especially with encoding which you would often just run as a
script.

It is quite some work to implement this feature and I am afraid that
extremely few users will be using it. So unless you convince other
users that your idea is needed I so not see it being implemented.

With --logfile, --progress or -u you can already watch the progress.


/Ole