parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: parallel: This should not happen. You have found a bug.


From: Linda Walsh
Subject: Re: parallel: This should not happen. You have found a bug.
Date: Sun, 11 Aug 2013 14:42:23 -0700
User-agent: Thunderbird



Ole Tange wrote:
On Sun, Aug 11, 2013 at 12:28 AM, Linda Walsh <gnu@tlinx.org> wrote:

I was trying to gunzip a bunch of small files ;
tried

for i in *.gz;do
sem -j6 $i
done

You seem to have forgotten gunzip:

  for i in *.gz;do
    sem -j6 gunzip $i
  done
---
oops yup
This ought to work just fine.

Personally I hate introducing an unneeded variable, so I would write it as:

  parallel gunzip ::: *gz
----
specialized syntax -- didn't know...

but your solution should work, too.

so then tried putting the sem in background

That would be wrong and can explain why you get fork-failed later: It
needs to be able to block if 6 jobs are already running.
----
I thought they might all use the same sem (probably not easy)... my problem was that when I did it the first way, it was periods of 100% cpu usage, with minor upticks, one for about 5 seconds up ~ 180%, but no periods of near 600% cpu usage which
I would see with a parallel make.


In the ProcessHacker, I would see 6 perl's launched at one time,
then see most or all complete in 1 refresh, see 1 running maybe
4-8s, then see some number (<=6) of perl's running. The 600 files...
have taken well over 10 minutes,
whereas I would have expected the same files processed in make or
xargs to take some number of seconds (<60)....


If you get the error on smaller/fewer files, please include those instead.
parallel: This should not happen. You have found a bug.
Please contact <parallel@gnu.org> and include:
* The version number: 20130122
* The bugid: Can't open semaphore file
/home/law/.parallel/semaphores/id-2fdev2fcons1.lock: Permission denied

Why is GNU Parallel not allowed to create
/home/law/.parallel/semaphores/id-2fdev2fcons1.lock?
----
Likely already in use?  I.e. It's cygwin on windows... so likely
if it is already open for write, it can't open it a 2nd time for
write...

This is explains what you see and is most likely the reason for the
failure you see.

Can you create the file by hand:

  echo > /home/law/.parallel/semaphores/id-2fdev2fcons1.lock
---
Yes...
While it is running (14 minutes and climbing), it seems to be testing
and recreating (if necessary) the semaphores directory under ".parallel".
I created my own dir, and it stayed, but if I removed the 'semaphores'
dir, it immediately re-created it...
It doesn't look like it is using real semaphores...but a file interlock
which might explain it running single threaded?
What OS is Perl running on and what file system is /home/law/.parallel
hosted on?
----
CYGWIN_NT-6.1-WOW64 / NTFS

(thats 32bit cygwin on 64-bit win7-sp1)...





reply via email to

[Prev in Thread] Current Thread [Next in Thread]