[Discuss-gnuradio] Re: synchronizing sound cards in a cluster (fwd)

[Eugen Leitl]

---------- Forwarded message ----------
Date: Thu, 13 Mar 2003 14:32:52 -0500 (EST)
From: Robert G. Brown <address@hidden>
To: Jim Lux <address@hidden>
Cc: address@hidden
Subject: Re: synchronizing sound cards in a cluster

On Thu, 13 Mar 2003, Jim Lux wrote:

> Anybody have any good ideas on how to synchronize the sampling from 
> multiple sound cards in a cluster using Ethernet as the interconnect. The 
> application would grab data from the sound card (notionally at 100 
> ksamples/second total, for two channels) and do a ton of signal 
> processing.  At some point in the processing, the streams of data need to 
> be shared between processors (i.e. to do beamforming), and so, needs to be 
> time registered.
> The bandwidth isn't a real challenge here (with, say, 16 processors, that's 
> only about 32 Mbps total), nor is latency, but synchronization is.
> 
> One can fairly easily synchronize to a millisecond over Ethernet, but this 
> application needs sync to, at worst, 1 sample time (20 microseconds) 
> although order of a microsecond would be nice.

a) Check out the documentation on http://www.ntp.org/documentation.html.
>From what it says, you can synchronize at roughly the level of network
latency with ntp alone, so you can (I would expect) get an otherwise
quiet LAN sync'd to a millisecond or even less.  NTP does correction
over a long time and damps to a common clock, so you MIGHT get down
below the 1 ms mark over time.  I doubt that ntp alone would make 10
usec.

b) Do you get to spend money?  Can you purchase each node its own GPS
clock?  The ntp docs suggest that if you have any sort of reference
clock (atomic, GPS, time pulse) your resolution is limited only by the
reference (and, probably things like gettimeofday, which are no better
than 2 usec as it is and can easily be worse).

c) If you don't get to spend money then you could TRY to use the onboard
tsc instead of coarsely adjusting the system clock per se.  I'm using it
as a timer in my benchmark code and can give you a wrapped assembler
fragment for reading it.  This clock is accurate to an inverse clock
(typically sub-nanosecond these days) BUT by the time you add the
overhead of reading it you diminish to perhaps 40-60 nanoseconds.
Still, in principle you have access to a clock with sub-usec resolution
(you can even measure and correct on average for the time required for
the wrapped call).

This clock is not configured for computing anything like absolute
systems time, so you'd have to do things like pingpong between systems
on the network a million times or so making slow adjustments to a
subtraction base until your "clocks" match within some resolution across
the entire network.

I actually don't think this would be horribly difficult.  It's sort of
like you and I looking at our watches and saying "10:02:02" (you adjust
a tick and say "10:02:04" (and I adjust a tick) and eventually we get to
the point where we are PREDICTING what the other person will say so that
given an average latency MEASURED to within some precision, we can say
that our clocks match within that precision.  I'm sure NTP does
something like this now with a coarser-grained clock, and you might be
able to steal it and just hack it to use the tsc and get what you want.

  rgb

-- 
Robert G. Brown                        http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:address@hidden



_______________________________________________
Beowulf mailing list, address@hidden
To change your subscription (digest mode or unsubscribe) visit 
http://www.beowulf.org/mailman/listinfo/beowulf