RE: [lwip-users] LWIP on a gigabit LAN

[Kieran Mansley]

On Tue, 2007-04-24 at 23:57 +0200, Jacob Gorm Hansen wrote:
> On Tue, 2007-04-24 at 14:50 -0700, Roger Cover wrote:
> > Sorry, 8 megabytes. 
> 
> In my first tests I seem to be getting around 25-30 megabytes per
> second. I suppose that to get better performance I need TCP Window
> Scaling and perhaps SACK. I found mention of SACK in LwIP here:
> http://edin.no-ip.com/html/?q=node/57 , but am not sure how to find the
> patch for it. Has anyone experimented with Window Scaling?

It's been a while since I used lwIP for high performance networking, but
this is essentially how I first got involved in the project, and it
formed the thesis of my PhD.  I was able to saturate a gigabit link, but
this was using a rather specialised piece of hardware.

To get high performance from lwIP you will need to use the raw API.  You
will also need plenty of memory (for buffers to allow streaming) and
plenty of CPU.  Make sure your driver is able to do things like
interrupt moderation and does not induce any extra copies.  You could
modify lwIP to do a copy and checksum in a single operation to improve
efficiency.  In terms of high performance tuning, lwIP is not really
that different to any other stack.  For TCP you need to configure large
windows, large MTUs, and do everything you can to avoid any loss.  SACK
will help a little in the presence of loss, but not a great deal.
Window Scaling is useful to get larger windows.  

Using Xen throws a whole extra dimension of problems at you.  You may
find that the overhead of forwarding interrupts to your domain for
example is reducing your performance.  Xen can saturate a gigabit link,
but as I understand it, it uses more CPU than native linux would do.

Kieran