GNU Parallel Bug Reports : The --eta option can lose input records

[David Welch]

The --eta option loses the current input record when printing a warning that the input records are taking a long time to read.

parallel --version

GNU parallel 20170422

Example:

(echo a; sleep 20s; echo b) | parallel --eta echo 2>/dev/null

Expected output:

a

b

Actual output:

a

Patch:

Index: parallel-20170422/src/parallel

===================================================================

--- parallel-20170422.orig/src/parallel

+++ parallel-20170422/src/parallel

@@ -6509,6 +6509,7 @@ sub total_jobs {

     my $record_queue = $self->{'commandlinequeue'}{'arg_queue'};

     my $start = time;

     while($record = $record_queue->get()) {

+ push @arg_records, $record;

  if(time - $start > 10) {

     ::warning("Reading ".scalar(@arg_records).

       " arguments took longer than 10 seconds.");

@@ -6517,7 +6518,6 @@ sub total_jobs {

     $opt::shuf && ::warning("Consider removing --shuf.");

     last;

  }

- push @arg_records, $record;

     }

     while($record = $record_queue->get()) {

  push @arg_records, $record;

Index: parallel-20170422/src/sem

===================================================================

--- parallel-20170422.orig/src/sem

+++ parallel-20170422/src/sem

@@ -6509,6 +6509,7 @@ sub total_jobs {

     my $record_queue = $self->{'commandlinequeue'}{'arg_queue'};

     my $start = time;

     while($record = $record_queue->get()) {

+ push @arg_records, $record;

  if(time - $start > 10) {

     ::warning("Reading ".scalar(@arg_records).

       " arguments took longer than 10 seconds.");

@@ -6517,7 +6518,6 @@ sub total_jobs {

     $opt::shuf && ::warning("Consider removing --shuf.");

     last;

  }

- push @arg_records, $record;

     }

     while($record = $record_queue->get()) {

  push @arg_records, $record;