Re: [Discuss-gnuradio] Multiple USRPs

[Martin Dvh]

Elaine Garbarine wrote:
> Martin,
> 
> I appreciate your response.  My application will indeed need full phase
> coherency.  Could you tell  me a bit more about how you achieved this?  I'm
> curious to know to what level you've been able to achieve phase
> synchronization?  More importantly, are any phase
> differences that do exist between signals received over multiple
> USRPs constant, or it time varying?
If the phase-difference between the signals is constant, the phase-difference 
you will see in gnuradio will be constant.
That is the whole point of this setup, measure or use phase and/or time 
differences.

In the current state, I see a phase (actually time)  difference of zero or one 
sample. (This is after decimation)
This has to do with how the usrps are synchronised.
I use a master-slave setup and the slave can be zero or one clock behind when 
it starts.

If I have the one sample phase-difference, this will stay constant.

I haven't measured subsample accuracy yet.

I am still struggling with the ddcs.
If I enable these I sometimes get bigger (although usually still constant) 
phase-difference.

What I did to achieve this was:
I made a master-slave setup and clock-locked the usrps.
I export the 64 MHz clock from the master usrp and input this as clock for the 
slave usrp.
I connect a flatcable from io-0..7 from the dboard in slot RXA in the master 
usrp to io-8..15 in dboard in RXA in the slave usrp.
Over this connection I output a single sync pulse.
(I don't touch the optional reference clock on io0)

I implemented a 32 bit counter in the fpga.
This is sent along with the samples to the host as an extra channel on channel 
0 (I interleave the 32 bits as two 16 bits numbers)
The normal samples are on channel 1, 2 (and 3 although this is not needed)
I implemented an align block in gnuradio.
This takes all the (interleaved short) channels from the usrps (interleaved 
short I,Q and interleaved short MSB/LSB 32 bit counter)
It then determines the difference in samplenumbers and skips enough samples on 
each input to align the samples.
(I only determine the difference once in a block of samples to keep the 
processing load low)

To get a single startpoint I implementeded a sync.
The gnuradio code calles a sync() function on the master usrp.
The master usrp outputs this sync on an io-line.
The slave receives this on an io line.
When a sync is received (from the software or from the input line)
The samplecounter is reset.
The phase of the ddcs is reset
The sample_strobe counter is reset
The rxbuffer is reset (this is needed because the sample_strobe is reset, took 
me a while to find this out)

You can send multiple syncs but only one is needed.

All is setup with a new MASTER_SLAVE register in the fpga
(SYNC,MASTER_ENABLE, SLAVE_ENABLE)

I get a perfect alignment (phase difference zero) or a one sample difference.
(I think this is from the slave seeing the sync pulse one clocktick later as 
the master)

This setup does have a drawback, though.
Because you need 3 channels  (RXA,RXB, samplecounter) you need to enable 4 
channels.
(can't use three channels, the way samples are aligned right now)
So you halve the maximum usb throughput. (4 channels versus 2 channels)

The other drawback is that you can only send a sync after the flowgraph has 
started running.
This is because the samplecounters are kept at zero if the usrp is not running, 
disabled or reset.
So you have to do some tricks to send the sync if you are running the wxgui.

The good thing is that it keeps aligned, even if you overload the usb 
connection and miss random sample blocks.

You can also use this setup with more then two usrps.
You can make a special cable which connects the master to all slaves.
But you can also make every slave also a master for a slave-slave, and so on.

I think it could be improved if I would only send the samplenumber once in 
every block that is sent from the fpga to the usb bus.
This would however have an impact on a lot more code.


Greetings,
Martin


> 
> If any of my questions are unclear just let me know.
> 
> Thanks again for the help,
> Elaine G.
> 
> On 12/6/05 2:52 PM, "Martin Dvh" <address@hidden> wrote:
> 
> 
>>Elaine Garbarine wrote:
>>
>>
>>>Hello all,
>>>
>>>I'm involved in a project to design a hardware testbed for Multiple-input
>>>Multiple-output systems.  I was considering GNU Radio and the USRP to
>>>accomplish this task.  I was wondering if any of you have connected multiple
>>>USRPs to build an RF system larger than 2 input 2 output (I would
>>>specifically like a 4X4 RF system).  If you have connected multiple USRPs
>>>together how exactly have you gone about it?
>>
>>I have been working on a system with two usrps for a total of 4 receive
>>channels.
>>For just 4 seperate receive or transmit channels, it is simple.
>>Connect two usrps to the host PC.
>>open multiple usrps in your python script and you can do anything you want
>>with it:
>>
>>#!/usr/bin/env python
>>
>>from gnuradio import gr
>>from gnuradio import usrp
>>from gnuradio.eng_option import eng_option
>>from optparse import OptionParser
>>
>>def main ():
>>     usrp0 = usrp.source_c (0, decimation, nchannels, mux, 0) #first parameter
>>is which usrp you want to open
>>     usrp1 = usrp.source_c (1, decimation, nchannels, mux, 0)
>>     dst00 = gr.file_sink (gr.sizeof_gr_complex, "file00.dat")
>>     dst01 = gr.file_sink (gr.sizeof_gr_complex, "file01.dat")
>>     dst10 = gr.file_sink (gr.sizeof_gr_complex, "file10.dat")
>>     dst11 = gr.file_sink (gr.sizeof_gr_complex, "file11.dat")
>>     fg.connect ((usrp0,0), dst00)
>>     fg.connect ((usrp0,1), dst01)
>>     fg.connect ((usrp1,0), dst10)
>>     fg.connect ((usrp1,1), dst11)
>>     fg.start ()
>>     raw_input ('Press Enter to quit: ')
>>     fg.stop ()
>>
>>if __name__ == '__main__':
>>     main ()
>>
>>For the project I am working on I need full phase coherency.
>>So I had to lock all the clocks/oscillators and synchronise and align all the
>>sample_data paths.
>>This required using a single 64 MHz clock for all usrps.
>>I also used hardware syncronisation between the usrps using a flatcable on the
>>io-pins on the daughtercards.
>>I modified the fpga firmware for it and built and used a software align block
>>in gnuradio.
>>
>>This work is allmost finished now (only receive channels).
>>If you also need full phase coherency,please tell.
>>
>>greetings,
>>Martin
>>
>>
>>>Thank you in advance for any assistance,
>>>Elaine Garbarine
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>_______________________________________________
>>>Discuss-gnuradio mailing list
>>>address@hidden
>>>http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>>
>>
>>
> 
> 
>