Re: Feature request: immediate output with -k option

[Warren Gish]

Thanks for the suggestion, Ole.
In a trial of processing a medium sized dataset, speed was reduced by 20% 
(mostly an increase in system CPU time) when using —lb.
(This is on an M3 Max.)

Significantly higher speed is preferred over the occasional benefit of 
immediate output when using —lb.

No buffering of the first job would perhaps increase speed a bit, while 
providing immediacy of output and avoiding the slowness of line buffering.

Thanks for your consideration.

—Warren


> On Jul 15, 2024, at 8:04 AM, Ole Tange <ole@tange.dk> wrote:
> 
> On Sat, Jun 15, 2024 at 9:09 AM Warren Gish <warrengish@gmail.com> wrote:
>> 
>> I often use “parallel -k” to maintain output order.  When using -k, I've 
>> noticed that no output is available from the parallel command until after 
>> the first job, which may be large, completes.
>> 
>> It would be helpful if the output from the first job could be made available 
>> immediately without buffering.
> 
> Use --lb
> 
> slowseq() { seq $@ | pv -qL 40 ;}
> export -f slowseq
> parallel --tag --lb -k 'slowseq {} 10000'  ::: 9900 9800 9700
> 
> --tag is only included to make it more visible what is happening.
> 
> /Ole