Re: plz help with tcpserver

[Alex fxmbsw7 Ratchev]

and if the reader has no idea what a gawk -v RS=\\n\\n 1 is there wont
be useful input for this thread by this reader, ..

On Mon, Jul 26, 2021 at 10:32 PM Alex fxmbsw7 Ratchev <fxmbsw7@gmail.com> wrote:
>
> sorry, run my gawk, not awk
> i didnt write sh, did i
>
> or at least as i did with -v RS=\\n\\n
>
> in the end my gawk, to explain, merges two following newlines to one,
> in a print process, this is short to test, and it failed
> so its a gawk bug ?
>
> On Mon, Jul 26, 2021 at 10:27 PM Greg Wooledge <greg@wooledge.org> wrote:
> >
> > On Mon, Jul 26, 2021 at 09:46:38PM +0200, Alex fxmbsw7 Ratchev wrote:
> > > > term 1
> > > > tcpserver 0 2000 strace -f gawk -v RS=\\n\\n 1 &
> > > >
> > > > strace for seeing weirdness
> > > > then
> > > > term 2, no printf or pipe, non eof input
> > > > cause, http clients dont end conn after printing req headers
> > > > ( nc 0 2000 to connect via netcat .org or so telnet client to localhost 
> > > > port 2000 )
> > > >
> > > > nc 0 2000
> > > >
> > > > then see strace began to show, read(
> > > >
> > > > then enter newlines some, it never prints, till control-c of nc
> > > > and result, the printed input, wont ever be recieved
> >
> > Pretend for a moment that the reader has no idea what "gawk -v RS=\\n\\n 1"
> > is supposed to do.
> >
> > Let's first verify how our tools actually *work*, shall we?  I don't
> > think you've programmed correctly for the line ending issue.
> >
> > ===== BEGIN EXAMPLE ONE =====
> > term1> tcpserver 0 2000 bash -c 'while read -r; do printf "Got: <%s>\n" 
> > "$REPLY"; done'
> >
> > term 2> telnet localhost 2000
> > ... Escape character is '^]'.
> > hello
> > >ot: <hello
> > goodbye
> > >ot: <goodbye
> > ===== END EXAMPLE ONE =====
> >
> > What happened?  The lines that I typed interactively in telnet were
> > received and processed in real time by the program that tcpserver
> > launched, which is good.  But there's an obvious carriage return issue
> > going on there.  You can see it by the greater-than signs overwriting
> > the first character of the line, instead of being at the end of the line.
> >
> > Remember, when you receive lines of text from a network socket, they may
> > be encoded with CR-LF line ending characters.  They're "supposed" to be
> > that way, because that's the "standard".  And it appears that telnet on
> > my system is enforcing that standard.
> >
> > We can verify that it's telnet doing so, and not tcpserver, by running
> > another experiment.
> >
> > ===== BEGIN EXAMPLE TWO =====
> > term1> same as before
> >
> > term2> { echo hello; sleep 1; echo goodbye; } | nc -N localhost 2000
> > Got: <hello>
> > Got: <goodbye>
> > ===== END EXAMPLE TWO =====
> >
> > That looks like what I was expecting to see!  So, we can conclude that
> > telnet was adding carriage returns to what I was typing, but nc -N
> > (using the implementation of nc on my system -- remember, nc is NOT
> > standard, and there are numerous variants of it out there) passed the
> > input through without modification.
> >
> > So, let's rewrite our server side, to handle the possible presence of
> > carriage returns in the input.  This is a pain in the ass to write as
> > a shell one-liner with bash -c, so I'll use a script.
> >
> > ===== BEGIN EXAMPLE THREE =====
> > term1> cat foo
> > #!/bin/bash
> > while read -r; do
> >   REPLY=${REPLY%$'\r'}
> >   printf 'Got: <%s>\r\n' "$REPLY"
> > done
> > term1> tcpserver 0 2000 ./foo
> >
> > term2> telnet localhost 2000
> > ... Escape character is '^]'.
> > hello
> > Got: <hello>
> > goodbye
> > Got: <goodbye>
> > ===== END EXAMPLE THREE =====
> >
> > All is well.  Now we know that two combinations of tools
> > (tcpserver, bash, telnet) and (tcpserver, bash, nc -N) both work fine.
> >
> > Now let's introduce awk.  Since I don't know what your thing does or is
> > trying to do, I'll try to duplicate the bash script that I just wrote,
> > using awk.  Let's start out simple:
> >
> > ===== BEGIN EXAMPLE FOUR =====
> > term1> tcpserver 0 2000 awk '{print "Got <" $0 ">"}'
> >
> > term2> telnet localhost 2000
> > ... Escape character is '^]'.
> > hello
> > goodbye
> > ^]
> > telnet> q
> > ===== END EXAMPLE FOUR =====
> >
> > Well, that was a complete failure, wasn't it?  Maybe that just
> > demonstrates that I don't know awk very well.  Maybe it's a buffering
> > thing (BashFAQ 009).  The FAQ page says that for gawk, we have to use
> > the fflush() function.  So let's try again:
> >
> > ===== BEGIN EXAMPLE FIVE =====
> > term1> tcpserver 0 2000 awk '{print "Got <" $0 ">"; fflush()}'
> >
> > term2> telnet localhost 2000
> > ... Escape character is '^]'.
> > hello
> > >ot <hello
> > goodbye
> > >ot <goodbye
> > ===== END EXAMPLE FIVE =====
> >
> > Looks familiar!  My awk program isn't handling the carriage returns from
> > telnet correctly.  That's a bug -- not in awk, but in my program, because
> > a network service has to be able to handle them.  They may be there, or
> > they may not.  But the good news is that, apart from the CR issue, it
> > worked as expected.
> >
> > Now let's move on to nc -N.
> >
> > ===== BEGIN EXAMPLE SIX =====
> > term1> same as above
> >
> > term2> { echo hello; sleep 1; echo goodbye; } | nc -N localhost 2000
> > Got <hello>
> > Got <goodbye>
> > ===== END EXAMPLE SIX =====
> >
> > Looks reasonable to me.
> >
> > Well, you're the awk guy, not me, so you can take it from here.  Figure
> > out how to write your program so that it correctly handles CR-LF or LF
> > line endings, use the fflush() command, and use nc -N on the client side if
> > you're on the same version of netcat that I am (netcat-openbsd 1.217-3).
> > Or if you're on a different version of netcat, figure out how to make yours
> > close cleanly when stdin hits EOF.
> >
> > Good luck.
> >