It is hard to
say when all we have is just the error message.
However, from
the error message, it tells that something apparently created
very large forces in the simulation and yielded a very high
velocity. That can happen either when the time step is too
large, or when particles overlap very much.
You said that
you reduced the time step, but the error still occurred. How
much did you reduce the time step? As a suggestion, start with
making the time step really small, like 0.001 times the time
step you had so far, and try again. If it works now, you can
increase it again.
Another thing
to look for is the initial forces. How did you set up the
system? If you set up the system randomly, there is a good
chance that some particles are very close to each other.
Depending on the interaction potential that you use, this
might result in very large forces. To get rid of these large
initial forces, you can use a warm-up with force capping
(inter forcecap) to get the system to a state where these
large forces disappear.
Olaf