[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
parallel interprets rsync log file format options as files to transfer t
From: |
Jason Paul Joines |
Subject: |
parallel interprets rsync log file format options as files to transfer that do not exist |
Date: |
Thu, 11 May 2017 11:39:48 -0400 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 |
GNU Parallel Users,
I am using GNU Parallel version 20170422 on Scientific Linux
release 6.6. When using rsync log file format options with parallel,
the logs are filled with errors because each log file format option gets
interpreted as a file to transfer that does not exist.
paralleltest/sdir$ parallel --version
GNU parallel 20170422
Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017
Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.
Web site: http://www.gnu.org/software/parallel
When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --citation'.
paralleltest/sdir$
paralleltest/sdir$ ls
tfile1 tfile2 tfile3 tfile4 tfile5 tfile6 tfile7 tfile8
paralleltest/sdir$
This works fine:
paralleltest/sdir$ ls -1 | parallel rsync -aHAX --delete --log-file
../testlog {} ../ddir/
paralleltest/sdir$
This also works fine, but with more detailed logging:
paralleltest/sdir$ for J in *; do rsync -aHAX --delete --log-file
../testlog --log-file-format="%o %f %L %l %b %i %t" "$J" ../ddir/; done
paralleltest/sdir$
This breaks because the rsync log file format options are
interpreted as files to transfer when passed through parallel:
paralleltest/sdir$ ls -1 | parallel rsync -aHAX --delete --log-file
../testlog --log-file-format="%o %f %L %l %b %i %t" {} ../ddir/
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%f" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%L" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%l" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%b" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%i" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%t" failed: No
such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1039) [sender=3.0.6]
The seven lines above are repeated for each of the eight files
in the transfer list.
I get the same behavior if I enclose the options in single
quotes instead of double quotes, put the log-file-format option before
the log-file option, or pass the the transfer list into a BASH script
that contains: parallel -j 8 rsync -aHAX --delete --log-file-format='%o
%f %L %l %b %i %t' --log-file ../testlog {} ../ddir/ ::: $@
Any ideas how to get parallel and rsync log-file-format options
to cooperate?
Thanks,
Jason
===========
- parallel interprets rsync log file format options as files to transfer that do not exist,
Jason Paul Joines <=