[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Out of memory when doing --pipe --block ?

From: hubert depesz lubaczewski
Subject: Re: Out of memory when doing --pipe --block ?
Date: Fri, 9 Feb 2018 09:45:25 +0100
User-agent: Mutt/1.5.23 (2014-03-12)

On Thu, Feb 08, 2018 at 07:50:22PM +0100, Ole Tange wrote:
> In practice this means you need 2.3 GB of free RAM and 3.8 GB of free
> virtual memory (RAM+swap) to run this.
> Given your output from 'free' I think that if you add 2 GB of swap,
> then that will solve your issue. Just because Perl loves swap, it does
> not mean that it will start swapping - it just wants to feel safe that
> it could if it wanted to. I have seen the same behaviour of other
> programs, so my guess is that you might get more performance from your
> system simply by adding a little swap space.
> The other option is to use --cat as described on

Yes, but --cat creates temp files, which I can't have as it slows down
things considerably.

Anyway - looks like I simply can't use parallel. It's memory
requirements are (from my perspective) impossible to handle, and I still
don't understand what exactly is the benefit of buffering stuff in


reply via email to

[Prev in Thread] Current Thread [Next in Thread]