parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

parallel cat


From: Dan Kokron
Subject: parallel cat
Date: Fri, 15 Jul 2011 14:39:38 -0400

All,

I have a bunch (~200) small (1K to 100K) binary files that I want to
'cat' into a larger file.  I usually use "cat pe* > diag", but this
takes considerable time on the Lustre file system we are using.  I am
exploring using GNU parallel for this task but have run into some
difficulties.  Basically the resulting diag file only contains one of
the input files.

I've tried the following variations.

parallel "cat {} >diag_amsua_n18_03.2011041700" ::: pe*
parallel cat {} ">"diag_amsua_n18_03.2011041700 ::: pe*
ls pe* | parallel cat {} ">"diag_amsua_n18_03.2011041700
ls pe* | parallel -j4 -k cat {} ">"diag_amsua_n18_03.2011041700
ls pe* | parallel -k cat {} ">"diag_amsua_n18_03.2011041700
parallel -j4 -k "cat {} >diag_amsua_n18_03.2011041700" ::: pe*

Should this work?  Any ideas what I'm doing wrong?
Thanks

parallel --version
GNU parallel 20110622

on
SLESS11

perl --version
This is perl, v5.10.0 built for x86_64-linux-thread-multi
-- 
Dan Kokron
Global Modeling and Assimilation Office
NASA Goddard Space Flight Center
Greenbelt, MD 20771
address@hidden
Phone: (301) 614-5192
Fax:   (301) 614-5304




reply via email to

[Prev in Thread] Current Thread [Next in Thread]