Re: GNU Parallel seems to drop data

[Ole Tange]

On Tue, Sep 25, 2012 at 3:48 PM, Dirk Eddelbuettel <edd@debian.org> wrote:

> Yes, now that I am in the office and my actual data, that verification in the
> next step. I probably also need the '-k' switch [ does that have "significant"
> performance implications? ] to ensure the order is the same which is important
> for the subsequent "munging" of the appropriately split files.

The penalty for -k is <1 ms CPU time, few kb RAM + 2 filehandles for
each job output that is postponed. In other words for all normal jobs
there is no performance penalty by using -k.

But remember: -k only affects stdout/stderr. It does *not* affect
output into files (like you are doing).


/Ole