Re: [lwip-users] Raw TCP - PBuf segment not decreasing

[Sergio R. Caprile]

I understand your frustration but it is vendor generated, not lwIP's
fault. Say "bye, thanks" to your vendor "docs" (or the lack of them)
and dive here:
        https://www.nongnu.org/lwip/2_1_x/      (or the version you're using)
The official source code tree has a directory named "apps" where you can
find working apps that will let you understand how to develop your own.
        https://www.nongnu.org/lwip/2_1_x/files.html
When in doubt, check how the gurus did it. SMTP is quite a piece.
Then there is the contrib tree, which carries all that stuff that has
been contributed but has not been elevated to the status of official app
(yet). Many of them are great starting points, particularly the
"tcpecho_raw" server, it is simple enough to grasp the idea.
        http://download.savannah.nongnu.org/releases/lwip/
... and finally the "examples" directory there has, well, LOTS of examples.

Regarding your "locks", I think you know what you are doing, and if you
did it right it should work, but we mortals tend to make mistakes and I
prefer to let the interrupt magic for a later time after all that is
happening has its proper explanation. So, if I were in your shoes, I
would signal on Ethernet Rx and Timer Tick and collect and send on main
loop; at least for now. For greater receive throughput you can queue on
Eth Rx and unqueue on main loop, the pbuf alloc and free functions can
be called from both contexts according to the docs and to the guys here
(I'm still flagging but I don't need no rx throughput for what I do
right now; a main loop with nothing else to do can tx/rx around 800KB+/s
on a 70MHz Cortex-M3 with no DMA, so don't rush to build a queue now)
Strange things like what you describe are easily explained by
reentrability issues, so...
You can play with sockets if you want, though they require an OS and
more memory.

Finally, tcp_write() just writes to a buffer; TCP will send when it
wants to.
Then tcp_output() is a way to suggest TCP to do it.
Calling tcp_write at the timer tick and not in the tcp_sent() callback
generates a dead time between the ACK and your timer, which can
translate in upto 50ms wasted. To keep TCP flowing, tcp_write() on
tcp_sent callback except if all the data for the former tick has been
sent; you will start again on next tick.
Before tcp_write(), you should (must) check for room with tcp_sndbuf().
It is OK to call tcp_output() after tcp_write() if you just pushed some
data. I never called tcp_output() just for the sake of it.
This is a high-performance example from the httpd server in times of
yore, haven't checked for recent incarnations.

    if (http_send(hs->pcb, hs)) {
      /* If we wrote anything to be sent, go ahead and send it now. */
      LWIP_DEBUGF(HTTPD_DEBUG | LWIP_DBG_TRACE, ("tcp_output\n"));
      tcp_output(hs->pcb);

and the send loop called by http_send() is something like this:
  do {
    LWIP_DEBUGF(HTTPD_DEBUG | LWIP_DBG_TRACE, ("Trying to send %d bytes
with flags: %u\n", len, apiflags));
    err = tcp_write(pcb, ptr, len, apiflags);
    if (err == ERR_MEM) {
      if ((tcp_sndbuf(pcb) == 0) ||
        (tcp_sndqueuelen(pcb) >= TCP_SND_QUEUELEN)) {
          /* no need to try smaller sizes */
          len = 1;
      } else {
        len /= 2;
      }
      LWIP_DEBUGF(HTTPD_DEBUG | LWIP_DBG_TRACE,
        ("Send failed, trying less (%d bytes)\n", len));
    }
  } while ((err == ERR_MEM) && (len > 1));

http_send() is called when starting to send "a file" and on the
connection's tcp_sent() callback.

OH! And there is the timing stuff... your main() should frequently call
the timer handling function that keeps all timed stuff (including TCP)
going:
        sys_check_timeouts()

Cheers