Re: Undetected fatal errors from redirected print

[arnold]

Hi Miguel.

Thanks for reporting an issue.

I think Andy has the right of it. The calls to fflush in close_rp are
only for two-way pipes, not regular files. Gawk relies on the regular
fclose to do the flush and return an error status if something went wrong.

As Andy showed, I decided quite long ago not to report close errors
as I got a lot of complaints about it.  That was in the early gawk 3.x
time frame, if not even earlier.

So, I don't see that there's really anything to change here.

Thanks again,

Arnold

"Andrew J. Schorr" <aschorr@telemetry-investments.com> wrote:

> Hi,
>
> A few thoughts about this:
>
> 1. If the file open fails, you get a fatal error. So for example, if you 
> don't have
> permission to open the output file, it fails:
>
> bash-4.2$ gawk 'BEGIN {print "hello" > "/dev/not.allowed"}'
> gawk: cmd. line:1: fatal: cannot redirect to `/dev/not.allowed': Permission 
> denied
>
> 2. In your case, the file open succeeded, but a subsequent write failed,
> which is pretty unusual unless a filesystem has filled up.
>
> If you look in io.c:close_redir, you'll see this:
>
> ...
>         status = close_rp(rp, how);
>
>         if (status != 0) {
>                 int save_errno = errno;
>                 char *s = strerror(save_errno);
>
>                 /*
>                  * BWK's awk, as far back as SVR4 (1989) would check
>                  * and warn about the status of close.  However, when
>                  * we did this we got too many complaints, so we moved
>                  * it to be under lint control.
>                  */
>                 if (do_lint) {
>                         if ((rp->flag & RED_PIPE) != 0)
>                                 lintwarn(_("failure status (%d) on pipe close 
> of `%s': %s"),
>                                          status, rp->value, s);
>                         else if ((rp->flag & RED_TWOWAY) != 0)
>                                 lintwarn(_("failure status (%d) on two-way 
> pipe close of `%s': %s"),
>                                          status, rp->value, s);
>                         else
>                                 lintwarn(_("failure status (%d) on file close 
> of `%s': %s"),
>                                          status, rp->value, s);
>                 }
>
>                 if (! do_traditional) {
>                         /* set ERRNO too so that program can get at it */
>                         update_ERRNO_int(save_errno);
>                 }
>         }
>
> So please try running again with --lint. I tested one case:
>
> bash-4.2$ strace -fo trace.txt -P answer.txt -e fault=write gawk --lint 
> 'BEGIN {print "42" > "answer.txt"; print close("answer.txt")}'
> strace: Requested path 'answer.txt' resolved into '/tmp/answer.txt'
> gawk: cmd. line:1: warning: failure status (-1) on file close of 
> `answer.txt': Function not implemented
> -1
>
> Which leads to:
>
> 3. If you really care about whether the I/O operations succeeded, you should 
> check
> the return code from the close() function call, which is -1, indicating an 
> error
> occurred.
>
> Regards,
> Andy
>
> On Mon, Nov 22, 2021 at 08:56:30PM -0500, Miguel Pineiro Jr. wrote:
> > While surveying the io error handling of a few awks, I found that gawk 
> > sometimes reports success upon a fatal failure to write a redirected print 
> > (or printf) statement's data.
> > 
> > Fedora 35 Worstation
> > Linux 5.14.17-301.fc35.x86_64
> > GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.1.0-p13, GNU MP 6.2.0)
> > GNU Awk, commit a08cc63b, 2021-09-09, the final version in master using 
> > Autoconf 2.69.
> > 
> > I tested those two versions. I'm assuming the issue still exists in master, 
> > but I did not confirm it. I did take a peek at `git diff a08cc63b..master 
> > interpret.h builtin.c io.c` and none of the few changes appear to be a 
> > remedy for the misbehavior I'm seeing.
> > 
> > My test script (which is at the end of this mail) tests the following 
> > commands:
> > 
> > gawk 'BEGIN {print "42" > "answer.txt"}'
> > gawk 'BEGIN {print "42" > "answer.txt"; close("answer.txt")}'
> > gawk 'BEGIN {print "42"}' > answer.txt
> > 
> > It uses strace to intercept and sabotage attempts to write the number 42 
> > into answer.txt. If all were well, gawk would emit an error message and 
> > return a non-zero exit status.
> > 
> > The first two commands have a redirected print statement. The first command 
> > leaves the closing of the stream to gawk. The second, closes the stream 
> > explicitly. In both cases, gawk fails to detect fatal io errors and returns 
> > success.
> > 
> > GAWK CMD: gawk 'BEGIN {print "42" > "answer.txt"}'
> > GAWK RET: 0
> > BYTES WRITTEN: 0
> > SYSCALL TRACE:  write(3, "42\n", 3)              = -1 ENOSYS (Function not 
> > implemented) (INJECTED)
> > 
> > GAWK CMD: gawk 'BEGIN {print "42" > "answer.txt"; close("answer.txt")}'
> > GAWK RET: 0
> > BYTES WRITTEN: 0
> > SYSCALL TRACE:  write(3, "42\n", 3)              = -1 ENOSYS (Function not 
> > implemented) (INJECTED)
> > 
> > I didn't debug, but I did look at the code for a bit. I think the problem 
> > is in close_rp. When it calls fclose/pclose on a redirected output stream 
> > with pending data in its buffer, it fails to discriminate between a fatal 
> > write(2) error during the flush and a subsequent, non-fatal close(2) error 
> > when freeing the underlying file descriptor. The fatal write errors then 
> > slip through as benign close errors.
> > 
> > Un-redirected output is not affected. On its way out, in close_io, gawk 
> > flushes stdout and stderr. If either flush fails, the failure is detected, 
> > a warning produced, and a non-zero status returned. All good. 
> > 
> > GAWK CMD: gawk 'BEGIN {print "42"}' > answer.txt
> > gawk: warning: error writing standard output: Function not implemented
> > GAWK RET: 1
> > BYTES WRITTEN: 0
> > SYSCALL TRACE:  write(1, "42\n", 3)              = -1 ENOSYS (Function not 
> > implemented) (INJECTED)
> > 
> > In case it's of use to anyone, my script follows. It requires strace and it 
> > will create two files in the working directory, answer.txt and trace.txt.
> > 
> > Take care,
> > Miguel
> > 
> > 
> > # Commands to be tested served on stdin
> > exec <<'EOF'
> > gawk 'BEGIN {print "42" > "answer.txt"}'
> > gawk 'BEGIN {print "42" > "answer.txt"; close("answer.txt")}'
> > gawk 'BEGIN {print "42"}' > answer.txt
> > EOF
> > 
> > # Sabotage gawk's attempt to write "42" to answer.txt.
> > while IFS= read -r gcmd; do
> >     echo
> >     >answer.txt >trace.txt
> >     echo GAWK CMD: "$gcmd"
> >     strace -fo trace.txt -P answer.txt -e fault=write sh -c "$gcmd"
> >     echo GAWK RET: $?
> >     echo BYTES WRITTEN: $(wc -c < answer.txt)
> >     echo SYSCALL TRACE: "$(sed -n '/write/s/^[^ ]*//p' trace.txt)"
> > done