parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Multiple sqlworkers


From: Thomas Bereknyei
Subject: Multiple sqlworkers
Date: Sun, 5 Apr 2020 23:21:05 -0400

Running some experiments with the sqlmaster/sqlworker system shows
that running multiple independent workers results in many jobs being
done multiple times. This is noted in the manual as

"If you have more than one --sqlworker jobs may be run more than once."

In simple testing, this is not an occasional thing, but results in
nearly all jobs being performed N times. It seems the table already
has the concept of jobs already taken (exitval: -1220) which would
suggest it should be locking each job. How can one create a system
where multiple workers grab jobs from the sql table in a way that
minimizes this? If this is a race condition, can they take jobs in a
shuffled/randomized order from available jobs in order to minimize
this?



reply via email to

[Prev in Thread] Current Thread [Next in Thread]