[lwip-users] Lwip tcp-stack reliability issue when using non-reliable network?

[Itzik Levi]

Hi All, 

Firstly, a bit of background:

I'm currently evaluating lwip stack in order to determine whether it can fit my needs.

The end-goal, is integrating Socket API --> TCP/IP --> PPPoS stack on both sides over a serial-like-interface(stream-based)  which isn't reliable.

I decided to unittest it before actually integrating the stack.

Setup:

(This is done symmetrically on both "client" and "server")

1. Network Interface:
1. The "physical layer" is currently a localhost connected tcp socket.
2. Using pppapi in order to create the network interface.
3. Its "output_cb" is the tcp connect socket(as mentioned before).
4. Using "pppos_input_tcpip" which is fed by the same tcp socket.
2. Forming a TCP connection over the interface:
1. Using lwip's bsd-like socket api.
2. Single socket connection.
3. Disabling Nagle.
4. Working with a non-blocking sockets.
5. Not writing and reading at the same time(mutex protected), but using lwip_poll in parallel to both.
3. Data validation:
1. Transferring the same generated data client-->server and server-->client and validating at the same time.

Results:

• When not introducing losses to the physical layer, the data is passed and validated successfully without any issues.
• When starting to randomly introduce losses to the physical layer(basically drop some data when output_cb is being called), it seems to go well for minutes, but eventually it appears that I'm loosing data when sending.

I'm sure I'm doing something wrong here, can you please help me figure this thing out?

Pointers to debug such a case is also welcome!

I'm attaching lwip's debug logs from both client and server, when losses are introduced to both physical layer ends.

 

In this specific case some data that was sent from the client --> server, and the server failed to validate the data.

 

The last received valid stream offset is 19008, the next 400 bytes were incorrect(future data), basically I got some skipped data.

lwip configuration:

SYS_LIGHTWEIGHT_PROT=1

NO_SYS =0

LWIP_TCPIP_CORE_LOCKING=1

LWIP_TCPIP_CORE_LOCKING_INPUT=1

TCP_WND=(100 * TCP_MSS)

TCP_SND_BUF=TCP_WND

MEMP_NUM_TCP_SEG=TCP_SND_QUEUELEN

I used lwip's repo commit: 5d8d21fcae63c36005baf1b15e91268836dec679.

(which is lwip 2.0.3 plus some..)

Please tell me if any more info is required.

Thanks,

Itzik

lwip_log_server.log.gz
Description: GNU Zip compressed data

lwip_log_client.log.gz
Description: GNU Zip compressed data