parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Help on For loop Rewrite


From: Blake, Lauren
Subject: Help on For loop Rewrite
Date: Mon, 12 May 2014 15:21:49 +0000

Hello,

I was hoping to get some guidance on a mistake that I am making in rewriting a for loop to perform the iterations of the loop in parallel using GNU Parallel. I have changed the for loop to the set-up that I found on many examples online, but I'm afraid that the syntax is not correct still, and I am still getting errors. 

When running my current edited script, I am getting errors saying "no file/directory *Name of file* exists" meaning that the commands that are supposed to create these files and directories are not being run properly. This leads me to believe that lines 15 - 19 of my edited (parallel) script For_loop_parallel.sh (attached) are not functioning properly.

On the man page, I examined the example they provided:
for x in `cat list` ; do
   no_extension=${x%.*};
   do_something $x scale $no_extension.jpg
   do_step2 <$x $no_extension
 done) | process_output
and how they changed it to this:

cat list | parallel "do_something {} scale {.}.jpg ; do_step2 <{} {.}" | process_output


But in that example, I do not understand how the parallel command will "know" to iterate through the x variable, as that disappeared in the parallel version. 

I believe my problem in my rewritten script just stems from my lack of understanding of how to tell GNU parallel what to iterate from. In my for loop, I am trying to iterate through each line using variable $line through a file called $fName (the Looped_through_file.txt), but currently don't see a way of including that in proper syntax for the parallel command. If you open up both of the For_loop scripts, you will see how I attempted to change the syntax of the (fully functioning) for_loop_regular script into the for_loop_parallel script based on the online examples. 

I did not include the large file that one of the commands references, as it is extremely large, but I did include the small file that is being referenced and is actually being looped through.

Any advice on properly rewriting my for loop so that the subsequent iterations can be sent to different cores will be very helpful. I have yet to see an example that maintains the iterated variable in the parallel command and do not understand how parallel can work without that information. However, I am new to Linux commands and programming in general, so perhaps there is some large aspect of Linux capabilities/syntax that I am unaware of. 

Thank you,
Lauren

Attachment: For_loop_parallel.sh
Description: For_loop_parallel.sh

Attachment: For_loop_regular.sh
Description: For_loop_regular.sh

Attachment: Looped_through_file.txt
Description: Looped_through_file.txt


reply via email to

[Prev in Thread] Current Thread [Next in Thread]