[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Processing files from a tar archive in parallel

From: Hans Schou
Subject: Re: Processing files from a tar archive in parallel
Date: Tue, 29 Mar 2011 23:41:41 +0200 (CEST)
User-agent: Alpine 2.00 (DEB 1167 2008-08-23)

On Tue, 29 Mar 2011, Ole Tange wrote:

While I loved Hans' idea, it does indeed have a race condition. This
should run 'ls -l' on each file after decompressing and clearly fails
now and then:

$ tar xvf ../i.tgz | parallel ls -l > ls-l
ls: cannot access 1792: No such file or directory

But you could unpack in a new dir and use:

A solution could be to print #1 file name when you got #2 file name. As tar files is a stream (sequence), #1 is written to disk when #2 name comes to the output. Something like:
  tar xvf x.tgz | ./ | parall....

Joke of the week:
You could also use strace and look for 'close', try:
  strace -eopen,close tar xf x.tgz

Pipe the output to a perl script and print the file name when you get the close command ;-)

Horsebakken 78, DK-2400 KĂžbenhavn NV

reply via email to

[Prev in Thread] Current Thread [Next in Thread]