Re: plz help with tcpserver

[Greg Wooledge]

On Mon, Jul 26, 2021 at 09:46:38PM +0200, Alex fxmbsw7 Ratchev wrote:
> > term 1
> > tcpserver 0 2000 strace -f gawk -v RS=\\n\\n 1 &
> >
> > strace for seeing weirdness
> > then
> > term 2, no printf or pipe, non eof input
> > cause, http clients dont end conn after printing req headers
> > ( nc 0 2000 to connect via netcat .org or so telnet client to localhost 
> > port 2000 )
> >
> > nc 0 2000
> >
> > then see strace began to show, read(
> >
> > then enter newlines some, it never prints, till control-c of nc
> > and result, the printed input, wont ever be recieved

Pretend for a moment that the reader has no idea what "gawk -v RS=\\n\\n 1"
is supposed to do.

Let's first verify how our tools actually *work*, shall we?  I don't
think you've programmed correctly for the line ending issue.

===== BEGIN EXAMPLE ONE =====
term1> tcpserver 0 2000 bash -c 'while read -r; do printf "Got: <%s>\n" 
"$REPLY"; done'

term 2> telnet localhost 2000
... Escape character is '^]'.
hello
>ot: <hello
goodbye
>ot: <goodbye
===== END EXAMPLE ONE =====

What happened?  The lines that I typed interactively in telnet were
received and processed in real time by the program that tcpserver
launched, which is good.  But there's an obvious carriage return issue
going on there.  You can see it by the greater-than signs overwriting
the first character of the line, instead of being at the end of the line.

Remember, when you receive lines of text from a network socket, they may
be encoded with CR-LF line ending characters.  They're "supposed" to be
that way, because that's the "standard".  And it appears that telnet on
my system is enforcing that standard.

We can verify that it's telnet doing so, and not tcpserver, by running
another experiment.

===== BEGIN EXAMPLE TWO =====
term1> same as before

term2> { echo hello; sleep 1; echo goodbye; } | nc -N localhost 2000
Got: <hello>
Got: <goodbye>
===== END EXAMPLE TWO =====

That looks like what I was expecting to see!  So, we can conclude that
telnet was adding carriage returns to what I was typing, but nc -N
(using the implementation of nc on my system -- remember, nc is NOT
standard, and there are numerous variants of it out there) passed the
input through without modification.

So, let's rewrite our server side, to handle the possible presence of
carriage returns in the input.  This is a pain in the ass to write as
a shell one-liner with bash -c, so I'll use a script.

===== BEGIN EXAMPLE THREE =====
term1> cat foo
#!/bin/bash
while read -r; do
  REPLY=${REPLY%$'\r'}
  printf 'Got: <%s>\r\n' "$REPLY"
done
term1> tcpserver 0 2000 ./foo

term2> telnet localhost 2000
... Escape character is '^]'.
hello
Got: <hello>
goodbye
Got: <goodbye>
===== END EXAMPLE THREE =====

All is well.  Now we know that two combinations of tools
(tcpserver, bash, telnet) and (tcpserver, bash, nc -N) both work fine.

Now let's introduce awk.  Since I don't know what your thing does or is
trying to do, I'll try to duplicate the bash script that I just wrote,
using awk.  Let's start out simple:

===== BEGIN EXAMPLE FOUR =====
term1> tcpserver 0 2000 awk '{print "Got <" $0 ">"}'

term2> telnet localhost 2000
... Escape character is '^]'.
hello
goodbye
^]
telnet> q
===== END EXAMPLE FOUR =====

Well, that was a complete failure, wasn't it?  Maybe that just
demonstrates that I don't know awk very well.  Maybe it's a buffering
thing (BashFAQ 009).  The FAQ page says that for gawk, we have to use
the fflush() function.  So let's try again:

===== BEGIN EXAMPLE FIVE =====
term1> tcpserver 0 2000 awk '{print "Got <" $0 ">"; fflush()}'

term2> telnet localhost 2000
... Escape character is '^]'.
hello
>ot <hello
goodbye
>ot <goodbye
===== END EXAMPLE FIVE =====

Looks familiar!  My awk program isn't handling the carriage returns from
telnet correctly.  That's a bug -- not in awk, but in my program, because
a network service has to be able to handle them.  They may be there, or
they may not.  But the good news is that, apart from the CR issue, it
worked as expected.

Now let's move on to nc -N.

===== BEGIN EXAMPLE SIX =====
term1> same as above

term2> { echo hello; sleep 1; echo goodbye; } | nc -N localhost 2000
Got <hello>
Got <goodbye>
===== END EXAMPLE SIX =====

Looks reasonable to me.

Well, you're the awk guy, not me, so you can take it from here.  Figure
out how to write your program so that it correctly handles CR-LF or LF
line endings, use the fflush() command, and use nc -N on the client side if
you're on the same version of netcat that I am (netcat-openbsd 1.217-3).
Or if you're on a different version of netcat, figure out how to make yours
close cleanly when stdin hits EOF.

Good luck.