[lwip-users] Best way to create a tunnel interface

[Paul Archer]

Hi all,

I am implementing a tunnel interface for lwIP and wanted to run this design by
the mailing list to see if anyone had any comments, or if someone had done
something similar in the past.

I'll try to describe the design I have come up with. Let me know what you think.

First some basics:
The tunnel will use a TCP socket to encapsulate the tunnel data.
The tunnel will have IP addresses in the 10.0.0.x range
The tunnel will run over an ethernet interface, en0 (192.168.0.x range).
I am running lwIP with an OS (FreeRTOS).

A quick description of the tunnel is, it uses a TCP socket to route through NAT
firewalls to connect to an outside server. This TCP socket will
encapulate TCP/IP and UDP/IP
data.

The TCP tunnel will have a small synchronisation header, and then the
tunnel IP packet payload.

What I have for my plan of a tunnel interface looks like this:

I have tried to include the data structure that will be used to communicate
between the layers in the braces eg, (pbuf)

On the Transmit side,
Application (socket) -> lwIP -> (pbuf) tunnel (socket)-> lwIP ->
(pbuf) ethernet -> TX

On the receive side,
RX -> Ethernet (pbuf) -> lwIP -> (socket) tunnel (pbuf) -> lwIP ->
(socket) Application.

This next paragraph is all to do with the tunnel section in the diagram above.
On the receive side I was planning on having a tunnel thread.
This thread would block on the TCP socket, strip off the
tunnel header, and push the IP data as a pbuf to lwIP.

On the transmit side, I was planning on either having a library call
for the lwiP output, that would
place the tunnel header on the IP packet, and then send the data to
the socket. The problem with this
I think, is that I am using lwIP to loop back around and block waiting
on lwIP (causing deadlock). To solve
this I am either thinking of implementing a packet queue, that could
be serviced by a tunnel worker thread, or
I send the data with a non-blocking socket send, but I am not sure if
this non-blocking send would
resolve the dead lock issue

A couple of concerns and thoughts:
Will a tunnel interface even work with lwIP (I can't see any reason why it wont)

I could probably use the netbuf API instead of the socket based API.
My concern with this is
how to align IP packets, and the holding of state through callbacks.
Does using the netbuf API solve the problem
of deadlock on the transmit side.

Dead locking the system, is my other concern with lwIP TCP thread
needing to communicate with the tunnel and
the tunnel needing to communicate with lwIP TCP thread also.

Does this sound like a solid design? Does a worker thread model look
like a cleaner implementation OR does
a netbuf API implementation with call backs look nicer. Which model
will solve the dead lock issue, either a queue based
model, with a worker thread servicing the queue, or is a library API
call to just include the tunnel header before sending to
the TCP socket OK.


-- 
----
Regards
Paul Archer
address@hidden