[lwip-devel] New TCP connection from out-of-window SYN in TIME-WAIT

[Angus Gratton]

Hi folks,

I wanted to propose a change in lwIP. I'm prepared to implement and submit a patch for this, but wanted to first check if it might be a change you'd consider.

Context:

Working on the MicroPython project (and previously ESP-IDF), we often run into issue reports where a device is handling frequent short lived TCP connections from a single client, and then suddenly a new connection stalls inexplicably. It comes up in other contexts as well.[1]

As I'm sure you're aware, what's happening is the client has reused a local port (I've observed this sometimes happens very quickly on desktop Linux), while a previous socket is still in TIME-WAIT.

AFAIK there's no bug there, everything lwIP does is RFC compliant (in particular, RFC1337). However, the experience is poor as a TCP connection suddenly stalls for an extended period of time (until the client times out, or lwIP expires TIME-WAIT on the old socket.)

Proposal:

Most other TCP/IP stacks (for example, OpenBSD[2] and Linux[3]) will treat a SYN with a higher sequence number received by a TIME-WAIT socket as a new connection. Would a patch for LWIP to do the same be considered?

I believe this change would be RFC compliant, as per RFC 1122 4.2.2.13 Closing a Connection.[4]

I don't think it creates additional real risk of TIME-WAIT assassination, as (IIUC) the socket would need to receive 2**31 bytes of data, be closed, and then a duplicate SYN packet that's somehow been delayed behind all the other traffic is delivered. At worst it seems consistent with the rest of lwIP's behaviour for sequence number wrapping (i.e. not implementing PAWS in an embedded environment).

It would add a little complexity and code size, and I don't know if that trade-off is worth it (or worth putting behind an option guard). One of the reasons that I wanted to ask before spending any more time on it.

Thanks for your time considering this, and for all your previous work on LWIP!

Cheers,

Angus

[1] For example, noticed https://lists.nongnu.org/archive/html/lwip-devel/2021-07/msg00002.html while searching the list.

[2] https://github.com/openbsd/src/blob/477ac4f17e6a6b377943ecf8e34bc324defc64a6/sys/netinet/tcp_input.c#L1226

[3] https://github.com/torvalds/linux/blob/master/net/ipv4/tcp_minisocks.c#L221

[4] https://www.rfc-editor.org/rfc/rfc1122.html#page-87