[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Processing files from a tar archive in parallel

From: Hans Schou
Subject: Re: Processing files from a tar archive in parallel
Date: Tue, 29 Mar 2011 18:02:28 +0200 (CEST)
User-agent: Alpine 2.00 (DEB 1167 2008-08-23)

On Tue, 29 Mar 2011, Benjamin R. Haskell wrote:

On Tue, 29 Mar 2011, Hans Schou wrote:
tar xvf big-file.tar.gz | parallel echo "Proc this file {}"

Hans, you left off the 'z' in 'tar zxvf':

No, its a new feature in tar (Ubuntu 10). If you specify a file, tar will test compression type it self and unzip it.

Try this at home:
 tar zcf htmls.tar.gz $(find /usr/share/doc -name '*.html')

And then do the thing:
 tar xvf htmls.tar.gz | parallel tidy -miq

Jay, you probably also want to 'rm' the files as you go, since space sounds like an issue.

And write a script which can handle one file, if you have a lot of commands to run on one file - or decitions.

Horsebakken 78, DK-2400 KĂžbenhavn NV

reply via email to

[Prev in Thread] Current Thread [Next in Thread]