Re: [Discuss-gnuradio] Multiple USRPs

[Martin Dvh]

Robert McGwier wrote:
> This is not quite right.  The DIFFERENT oscillators on the USRP boards
> have different frequencies and thus constantly changing phase angle with
> respect to each other and with probability 1, the frequencies of both
> oscillators will change with changing temp, air flow, etc..  The only
> way to avoid this is the buffer the clock from one board and use it to
> drive the other board.  
Maybe you missed this line:
>I export the 64 MHz clock from the master usrp and input this as clock for the 
>slave usrp.
So I do drive all boards from a common clock.

>The system will NOT be coherent unless it IS
> coherent.  ;-)
Well, I think it IS coherent.

Greetings,
Martin

.
> 
> Bob
> 
> 
> 
> 
> Martin Dvh wrote:
> 
>> Elaine Garbarine wrote:
>>  
>>
>>> Martin,
>>>
>>> I appreciate your response.  My application will indeed need full phase
>>> coherency.  Could you tell  me a bit more about how you achieved
>>> this?  I'm
>>> curious to know to what level you've been able to achieve phase
>>> synchronization?  More importantly, are any phase
>>> differences that do exist between signals received over multiple
>>> USRPs constant, or it time varying?
>>>   
>>
>> If the phase-difference between the signals is constant, the
>> phase-difference you will see in gnuradio will be constant.
>> That is the whole point of this setup, measure or use phase and/or
>> time differences.
>>
>> In the current state, I see a phase (actually time)  difference of
>> zero or one sample. (This is after decimation)
>> This has to do with how the usrps are synchronised.
>> I use a master-slave setup and the slave can be zero or one clock
>> behind when it starts.
>>
>> If I have the one sample phase-difference, this will stay constant.
>>
>> I haven't measured subsample accuracy yet.
>>
>> I am still struggling with the ddcs.
>> If I enable these I sometimes get bigger (although usually still
>> constant) phase-difference.
>>
>> What I did to achieve this was:
>> I made a master-slave setup and clock-locked the usrps.
>> I export the 64 MHz clock from the master usrp and input this as clock
>> for the slave usrp.
>> I connect a flatcable from io-0..7 from the dboard in slot RXA in the
>> master usrp to io-8..15 in dboard in RXA in the slave usrp.
>> Over this connection I output a single sync pulse.
>> (I don't touch the optional reference clock on io0)
>>
>> I implemented a 32 bit counter in the fpga.
>> This is sent along with the samples to the host as an extra channel on
>> channel 0 (I interleave the 32 bits as two 16 bits numbers)
>> The normal samples are on channel 1, 2 (and 3 although this is not
>> needed)
>> I implemented an align block in gnuradio.
>> This takes all the (interleaved short) channels from the usrps
>> (interleaved short I,Q and interleaved short MSB/LSB 32 bit counter)
>> It then determines the difference in samplenumbers and skips enough
>> samples on each input to align the samples.
>> (I only determine the difference once in a block of samples to keep
>> the processing load low)
>>
>> To get a single startpoint I implementeded a sync.
>> The gnuradio code calles a sync() function on the master usrp.
>> The master usrp outputs this sync on an io-line.
>> The slave receives this on an io line.
>> When a sync is received (from the software or from the input line)
>> The samplecounter is reset.
>> The phase of the ddcs is reset
>> The sample_strobe counter is reset
>> The rxbuffer is reset (this is needed because the sample_strobe is
>> reset, took me a while to find this out)
>>
>> You can send multiple syncs but only one is needed.
>>
>> All is setup with a new MASTER_SLAVE register in the fpga
>> (SYNC,MASTER_ENABLE, SLAVE_ENABLE)
>>
>> I get a perfect alignment (phase difference zero) or a one sample
>> difference.
>> (I think this is from the slave seeing the sync pulse one clocktick
>> later as the master)
>>
>> This setup does have a drawback, though.
>> Because you need 3 channels  (RXA,RXB, samplecounter) you need to
>> enable 4 channels.
>> (can't use three channels, the way samples are aligned right now)
>> So you halve the maximum usb throughput. (4 channels versus 2 channels)
>>
>> The other drawback is that you can only send a sync after the
>> flowgraph has started running.
>> This is because the samplecounters are kept at zero if the usrp is not
>> running, disabled or reset.
>> So you have to do some tricks to send the sync if you are running the
>> wxgui.
>>
>> The good thing is that it keeps aligned, even if you overload the usb
>> connection and miss random sample blocks.
>>
>> You can also use this setup with more then two usrps.
>> You can make a special cable which connects the master to all slaves.
>> But you can also make every slave also a master for a slave-slave, and
>> so on.
>>
>> I think it could be improved if I would only send the samplenumber
>> once in every block that is sent from the fpga to the usb bus.
>> This would however have an impact on a lot more code.
>>
>>
>> Greetings,
>> Martin
>>
>>
>>  
>>
>>> If any of my questions are unclear just let me know.
>>>
>>> Thanks again for the help,
>>> Elaine G.
>>>
>>> On 12/6/05 2:52 PM, "Martin Dvh" <address@hidden> wrote:
>>>
>>>
>>>   
>>>
>>>> Elaine Garbarine wrote:
>>>>
>>>>
>>>>     
>>>>
>>>>> Hello all,
>>>>>
>>>>> I'm involved in a project to design a hardware testbed for
>>>>> Multiple-input
>>>>> Multiple-output systems.  I was considering GNU Radio and the USRP to
>>>>> accomplish this task.  I was wondering if any of you have connected
>>>>> multiple
>>>>> USRPs to build an RF system larger than 2 input 2 output (I would
>>>>> specifically like a 4X4 RF system).  If you have connected multiple
>>>>> USRPs
>>>>> together how exactly have you gone about it?
>>>>>       
>>>>
>>>> I have been working on a system with two usrps for a total of 4 receive
>>>> channels.
>>>> For just 4 seperate receive or transmit channels, it is simple.
>>>> Connect two usrps to the host PC.
>>>> open multiple usrps in your python script and you can do anything
>>>> you want
>>>> with it:
>>>>
>>>> #!/usr/bin/env python
>>>>
>>>>     
>>>> from gnuradio import gr
>>>> from gnuradio import usrp
>>>> from gnuradio.eng_option import eng_option
>>>> from optparse import OptionParser
>>>
>>>   
>>>
>>>> def main ():
>>>>    usrp0 = usrp.source_c (0, decimation, nchannels, mux, 0) #first
>>>> parameter
>>>> is which usrp you want to open
>>>>    usrp1 = usrp.source_c (1, decimation, nchannels, mux, 0)
>>>>    dst00 = gr.file_sink (gr.sizeof_gr_complex, "file00.dat")
>>>>    dst01 = gr.file_sink (gr.sizeof_gr_complex, "file01.dat")
>>>>    dst10 = gr.file_sink (gr.sizeof_gr_complex, "file10.dat")
>>>>    dst11 = gr.file_sink (gr.sizeof_gr_complex, "file11.dat")
>>>>    fg.connect ((usrp0,0), dst00)
>>>>    fg.connect ((usrp0,1), dst01)
>>>>    fg.connect ((usrp1,0), dst10)
>>>>    fg.connect ((usrp1,1), dst11)
>>>>    fg.start ()
>>>>    raw_input ('Press Enter to quit: ')
>>>>    fg.stop ()
>>>>
>>>> if __name__ == '__main__':
>>>>    main ()
>>>>
>>>> For the project I am working on I need full phase coherency.
>>>> So I had to lock all the clocks/oscillators and synchronise and
>>>> align all the
>>>> sample_data paths.
>>>> This required using a single 64 MHz clock for all usrps.
>>>> I also used hardware syncronisation between the usrps using a
>>>> flatcable on the
>>>> io-pins on the daughtercards.
>>>> I modified the fpga firmware for it and built and used a software
>>>> align block
>>>> in gnuradio.
>>>>
>>>> This work is allmost finished now (only receive channels).
>>>> If you also need full phase coherency,please tell.
>>>>
>>>> greetings,
>>>> Martin
>>>>
>>>>
>>>>     
>>>>
>>>>> Thank you in advance for any assistance,
>>>>> Elaine Garbarine
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Discuss-gnuradio mailing list
>>>>> address@hidden
>>>>> http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>>>>
>>>>>       
>>>>
>>>>     
>>>
>>>
>>>   
>>
>>
>>
>>
>> _______________________________________________
>> Discuss-gnuradio mailing list
>> address@hidden
>> http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>
>>  
>>
> 
>