Re: How to test the output of gnu parallel

[Ole Tange]

On Tue, Sep 18, 2012 at 9:22 AM, yacob sen <yacob_123@yahoo.com> wrote:
>
> Dear All,
>
> I am trying to test if a file exists or not in a remote server. I have 4 
> processors on my disposal in my laptop. Here is what I am trying to do:
>
> for  value  in {10..20}
> do
> #*************************************************************************************************
> seq -w 1 1 100 | parallel  -j+0  wget ${ftp_site}/$value/{}/{}.F >& /dev/null 
>  && echo  Y >> tmp || echo  N  >> tmp
>
>           read test < tmp
>          if [ $test = "Y"  ];then
>            echo "to an output  file, "temp.$value.{}""
>          fi
> done
> #*************************************************************************************************

Why only use 4 processes in parallel? If the webserver can handle it
you can spawn more:

parallel -j100 "wget -q ${ftp_site}/{1}/{2}/{2}.F && echo {1}/{2}"
:::: <(seq 10 20) <(seq -w 100) > outfile

> If the file exists in the remote server, the value in "tmp" should get "Y" , 
> in that case the file is saved to a file.  My question is how to test the 
> existence of all the 4 processors at the same time ?

The problem here is that you need a tmp file for each process -
otherwise it will be overwritten and thus you cannot predict by which
one, so avoid the tmp-files if you can.


/Ole