Re: [lwip-users] Zero window and refused data problem

[Joel Cunningham]

Oleg,

I agree that setting the window update threshold below 1 MSS would be bad.  I didn't realize your window was being updated in such small amounts.  Is there a reason why your application isn't continuing to drain the buffer?

In terms of receiving a zero-window probe.  Is LwIP ACKing the probe because you technically do have window space available because your application read some data, but not enough to trigger the threshold?  If that's the case, then it sounds like LwIP is behaving correctly.  If LwIP is continuing to accept data even though the receive buffer is full and real window is 0, then that's a problem.

A zero window probe that is borrowing the probe byte of the next unsent segment is normal and in this case, the it sounds like the window has space, but we aren't advertising that space.  Eventually the window should fully fill up and LwIP would (hopefully) stop ACKing zero window probe.

Joel

On Nov 07, 2016, at 01:16 PM, Oleg Gladyshev <address@hidden> wrote:

Joel, thank you for your reply. 

Oleg,

In your original post, you mentioned that your application reads some data out of the buffer, but this doesn't result in a window update. This is because LwIP waits until 1/4 of the window has been opened up.  There is a setting that controls the update threshold in opt.h.  You can adjust this to match your use case.

Yes, it is normal behavior to aviod silly window if I right. I know that I can reduce threshold, but IMHO it is not good solution. I can change threshold from N to M bytes, but for M bytes the problem still here. In worst case I need exactly M-1 mbox capacity. Obviosly I can't set threshold to few bytes because it will disable silly window avoidance.

I'm also surprised that when LwIP ACKs the zero window probe, it doesn't contain the updated window value. 

Yes, it's true for 1.4.1 version. And it looks right for me because this time real (internal) window size still less than the threshold.

What top-level API is your application using, sockets or Raw?  With the Raw API, I believe you have to call tcp_recved() after your application receives data out of the buffer to update the window.

No, I use netconn API which calls tcp_recved() internally.

Joel

On Nov 07, 2016, at 10:03 AM, Oleg Gladyshev <address@hidden> wrote:

Hi Noam, 

Thank you very much for your reply but I think you misunderstood me. 
Look at the article: https://wiki.wireshark.org/TCP%20ZeroWindow
Of course my host (controlled by Windows now) sends data using quite large packets. But when device's buffer becomes full
host starts periodical checking of the buffer. Windows performs it using 1 byte packets called "Zero Window Probe packet" and we can't affect this behavior.

Hi,

First of all if you have control over the host machine avoid sending 1 byte. This is not efficient.
I suggest adding a dual mechanism. Defining size and/or timeout.

I mean adding some mechanism that counts bytes ready to be sent. If you have more than X
Bytes you send them out. Also add a timeout. Meaning if you have collected less then X bytes
but a time period has elapsed since the last send, send it anyway. This way you force the sender
to send a minimum sized buffer and not send out 1 byte at a time with a large overhead...

At your device side you probably have a DMA collecting data. You may modify the code and check
How many bytes have you received. Similar to the above mechanism. If you got less than some limit
you do not allocate a pBuf ... only if you have a minimum data size or a time difference from last
data handling you allocate a pBuf, copy data to it and let the TCP stack handle it.

BR,
Noam.

-----Original Message-----
From: lwip-users [mailto:address@hidden On Behalf Of Oleg Gladyshev
Sent: Monday, November 07, 2016 3:23 PM
To: address@hidden
Subject: [lwip-users] Zero window and refused data problem

    Hi all,
I develop a device which receives data from host machine using TCP (LwIP 1.4.1). Host sends data continuously queuing it to the controller.
Controller process data as fast as possible, but data process time is not determined. it can be from milliseconds to hours. And I wish to use TCP/IP incoming buffer as buffer for the data.

In my case TCP window on the device becomes full very often. Sometimes device's application receives from the TCP stack portion of data less than TCP windows size. This case LwIP doesn't update receive window leaving it zero. But the device able to receive some bytes now. And when the host sends 1 byte ZeroProbe packet device receives it.
For this one byte LwIP allocates new pbuf structure and sends it to application using sys_mbox_xxx call.
I use fixed-size mbox queues so after some probe packets from the host mbox becomes full and refuses new data. tcp_input notices this and tries to deliver refused data to the application every time it called.
But if application still refuses new data tcp_input() doesn't send ACK to the host! And host disconnects after some tries.

Making my mboxes dynamically doesn't solve the problem because in general we can waste all RAM with (TCP_WND-1) pbuf structures for every incoming byte from window probe packets.

I think it would be better to ACK with previous value to the host instead of silently dropping new data. What do you think about the issue? What is the best way to avoid disconnects in my project?

--
Best regards, Oleg

_______________________________________________
lwip-users mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/lwip-users