[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Multiple sqlworkers
From: |
Thomas Bereknyei |
Subject: |
Multiple sqlworkers |
Date: |
Sun, 5 Apr 2020 23:21:05 -0400 |
Running some experiments with the sqlmaster/sqlworker system shows
that running multiple independent workers results in many jobs being
done multiple times. This is noted in the manual as
"If you have more than one --sqlworker jobs may be run more than once."
In simple testing, this is not an occasional thing, but results in
nearly all jobs being performed N times. It seems the table already
has the concept of jobs already taken (exitval: -1220) which would
suggest it should be locking each job. How can one create a system
where multiple workers grab jobs from the sql table in a way that
minimizes this? If this is a race condition, can they take jobs in a
shuffled/randomized order from available jobs in order to minimize
this?
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Multiple sqlworkers,
Thomas Bereknyei <=