lwip-users
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [lwip-users] UDP-based reliable bulk data transfer


From: pupkin
Subject: Re: [lwip-users] UDP-based reliable bulk data transfer
Date: Thu, 27 Apr 2017 07:38:26 -0700 (MST)

I agree, dedicated link is generally quite reliable. I'm more concerned about
SW timing on both sides of the link. At 1Gb there is < 20us per packet.
Getting close to capacity requires careful attention to throttling and flow
control. If the rate is too slow I will under-utilize the bandwidth, if it
is too fast, I will start losing packets. If the feedback loop is too tight,
the flow is going to be jerky, if it is too sloppy, it will take longer time
to settle. I've seen the code to do these kind of things properly and it was
quite involved, I'd rather not reinvent this particular wheel if I can.

I'm looking at enet and it uses non-blocking UDP sockets with select().  I'm
running lwIp 1.4.1 on bare metal with NO_SYS=1 and I'm trying to find a way
to implement select() for the tx side. Unless I'm missing something obvious,
there appears to be no mechanism for the driver to signal tx completion, is
that right?

Regards,
VP


Patrick Klos-2 wrote
> On 4/25/2017 6:23 PM, Nenad Pekez wrote:
>> Hi Patrick,
>>
>> did you actually port one of these 2 proposed protocols to your 
>> embedded system or implemented your own UDP reliable protocol? Also, 
>> did you get bitrates close to 100Mb?
> 
> The point I was trying to make was that you barely need to do anything 
> to make a dedicated link reliable.  Some people may disagree, but I ran 
> a raw frame test for hours without loss.  That indicates to me that the 
> link itself is quite reliable.  Adding a sequence number, window and ACK 
> mechanism can be done quite simply. I was also saying that it would be a 
> simple protocol to implement, but I haven't implemented it myself (at 
> least not as a high speed data transfer protocol).
> 
> We had ENET on our device for other reasons, but have since removed it 
> and replaced it with a simpler "reliable UDP protocol" we wrote ourselves.
> 
> Patrick Klos
> Klos Technologies, Inc.
> 
>>> On 4/25/2017 3:16 AM, pupkin wrote:
>>> > Greetings,
>>> >
>>> > I need to send large quantities of data reliably from a Zynq-based 
>>> device to
>>> > a PC.
>>> > I'm looking for a data transfer protocol which can get as close as 
>>> possible
>>> > to wire speed on a dedicated 1Gb ethernet link. It would be nice 
>>> although
>>> > not strictly required it it worked over arbitrary internet
>>> connections.
>>> >
>>> > Currently I'm considering either porting http://enet.bespin.org/ or
>>> > http://udt.sourceforge.net/ to lwIp or rolling my own UDP-based 
>>> protocol. I
>>> > will need support for zero-copy with scatter-gather DMA.
>>> >
>>> > Has anyone done this before? Any suggestions?
>>> >
>>> > Regards
>>> > VP
>>>
>>> For what it's worth, I've done some tests on a dedicated 100Mb link and
>>> they are quite reliable. I suspect the same could be said about a
>>> dedicated 1Gb link? I ran a test once on a dedicated link between an
>>> embedded system and a Windows computer with no lost packets for hours
>>> (maybe even overnight - I forget the exact details).
>>>
>>> That means that you should be able to implement a high speed UDP based
>>> transfer protocol with very little overhead to make it "reliable". I
>>> would bet that creating a "window" of UDP packets (with sequence
>>> numbers) will do what you want with the receiver sending back an ACK of
>>> sorts for every so many packets or if it sees a sequence number missing.
>>>
>>> I'm not sure about the zero-copy scatter-gather support - you'll
>>> probably just have to try it. The lwIP stack might scan your pbuf and
>>> decide to copy the packet to a single buffer pbuf depending on the
>>> capabilities of the ethernet interface. Even so, your Zynq processor
>>> will probably be able to handle the copies faster than a 1Gb link could
>>> transfer the data?
>>>
>>> Good luck!
>>>
>>> Patrick Klos
>>> Klos Technologies, Inc.
> 
> 
> _______________________________________________
> lwip-users mailing list

> lwip-users@

> https://lists.nongnu.org/mailman/listinfo/lwip-users





--
View this message in context: 
http://lwip.100.n7.nabble.com/UDP-based-reliable-bulk-data-transfer-tp29424p29470.html
Sent from the lwip-users mailing list archive at Nabble.com.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]