Re: Feature request: immediate output with -k option

[Ole Tange]

On Mon, Jul 15, 2024 at 11:33 PM Warren Gish <warrengish@gmail.com> wrote:

> In a trial of processing a medium sized dataset, speed was reduced by 20% 
> (mostly an increase in system CPU time) when using —lb.

Using {= =} you have access to all internal variables in GNU Parallel
- even $opt::ungroup with determines if the output should be grouped
(buffered):

    parallel -k -u doit {= 'if(seq() > 1) { $opt::ungroup = 0 }' =} ::: 9 1 2 3

This is not officially supported. But I do not see anything in the
immediate future that would break this.

There are multiple considerations if this should be the default. The
most serious would be POLA
(https://en.wikipedia.org/wiki/Principle_of_least_astonishment).

* The first job will act differently from the rest:

    doit() {
      echo "$@" ERR >&2
      echo "$@" out
      sleep 0.$1
      echo "$@" ERR >&2
      echo "$@" out
    }
    export -f doit
    # STDERR/stdout is grouped together for jobs > 1 due to buffering
    parallel -k -u doit {= 'seq() > 1 and $opt::ungroup = 0' =} ::: 9 1 2 3

* The first job will have a tty, but the next ones will not.

I also have the feeling that enforcing this will cause other problems
down the line: In earlier versions job 1 would default to getting a
tty and that caused issues that were hard to debug because this
applied only to job 1.

Given there is a work-around I am reluctant to implement this.

I will, however, include the work-around in man parallel_examples; and
as long as it is in the examples, this use will be supported.


/Ole