[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

parallel cat

From: Dan Kokron
Subject: parallel cat
Date: Fri, 15 Jul 2011 14:39:38 -0400


I have a bunch (~200) small (1K to 100K) binary files that I want to
'cat' into a larger file.  I usually use "cat pe* > diag", but this
takes considerable time on the Lustre file system we are using.  I am
exploring using GNU parallel for this task but have run into some
difficulties.  Basically the resulting diag file only contains one of
the input files.

I've tried the following variations.

parallel "cat {} >diag_amsua_n18_03.2011041700" ::: pe*
parallel cat {} ">"diag_amsua_n18_03.2011041700 ::: pe*
ls pe* | parallel cat {} ">"diag_amsua_n18_03.2011041700
ls pe* | parallel -j4 -k cat {} ">"diag_amsua_n18_03.2011041700
ls pe* | parallel -k cat {} ">"diag_amsua_n18_03.2011041700
parallel -j4 -k "cat {} >diag_amsua_n18_03.2011041700" ::: pe*

Should this work?  Any ideas what I'm doing wrong?

parallel --version
GNU parallel 20110622


perl --version
This is perl, v5.10.0 built for x86_64-linux-thread-multi
Dan Kokron
Global Modeling and Assimilation Office
NASA Goddard Space Flight Center
Greenbelt, MD 20771
Phone: (301) 614-5192
Fax:   (301) 614-5304

reply via email to

[Prev in Thread] Current Thread [Next in Thread]