Re: [Diffutils-devel] diffutils-3.6.17-d5ba on HP-UX

[Bruno Haible]

Hi Jim,

> Unfortunately, that would reduce test coverage: of the case in which
> diff's stdin happens to be closed. As you know, we try hard not to
> dumb-down tests to accommodate inadequate test environments.

Summary of analysis:

1) The issue is not with the shell. The HP-UX /bin/sh (as well as the
   HP-UX /bin/ksh) support the <&- and >&- syntax, as defined in section
   2.7.5 of 
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

2) The issue is with the behaviour of 'diff' when file descriptor 0 is
   closed. The behaviour on HP-UX is correct, and the behaviour on Linux
   and the other platforms is incorrect. The test verifies the incorrect
   behaviour, and therefore fails when 'diff' behaves correctly.

Details:

The test scenario I used on Linux and HP-UX is:
- Preparation:
  $ cd tests
  $ echo a > a
- Command:
  $ ../src/diff -N - a <&-

Result on HP-UX:
$ ../src/diff -N - a <&-
diff: -: Bad file number

Result on Linux:
$ ../src/diff -N - a <&-
0a1
> a

I added a couple of fprintf statements at selected places, and got this
output:

On HP-UX:
file_block_read fd=0 size=65536 0
block_read: read(fd=0) -> -1
block_read failed
pfatal_with_name
error() 2 9 |%s|
diff: -: Bad file number

On Linux:
file_block_read fd=0 size=4096 0
block_read: read(fd=0) -> 2
block_read: read(fd=0) -> 0
file_block_read block_read returned 2
file_block_read fd=0 size=1 1

As you can see, the essential difference is how block_read behaves
when reading contents from the first argument, that is, stdin, which
ought to reference a closed file descriptor. On HP-UX, the read()
call fails with errno = 9 = EBADF, which is correct. On Linux, the
read() call succeeds and reads 2 bytes - from a supposedly closed
file descriptor!

Now let's look into more details into the system calls:

On HP-UX:

$ tusc ../src/diff -N - a <&-
execve("../src/diff", 0x87fffffffffffb08, 0x87fffffffffffb30) = 0 [64-bit]
...
ioctl(0, TCGETA, 0x87fffffffffff3e0) ..................... ERR#25 ENOTTY
fstat(0, 0x87fffffffffff420) ............................. = 0
clock_gettime(CLOCK_REALTIME, 0x87fffffffffff3f0) ........ = 0
stat("a", 0x87fffffffffff560) ............................ = 0
open("a", O_RDONLY|0x800, 0) ............................. = 3
brk(0x600000000005b000) .................................. = 0
read(0, 0x600000000004a780, 65536) ....................... ERR#9 EBADF
fcntl(1, F_GETFL, 1942) .................................. = 131074
getpid() ................................................. = 27608 (27607)
pstat(PSTAT_PROC, 0x87ffffffffffeea0, 1216, 0, 27608) .... = 1
diff: write(2, "d i f f :   ", 6) .............................. = 6
-write(2, "- ", 1) ........................................ = 1
: Bad fiwrite(2, ":   B a d   f i ", 8) .......................... = 8
le numbewrite(2, "l e   n u m b e ", 8) .......................... = 8
rwrite(2, "r ", 1) ........................................ = 1

write(2, "\n", 1) ........................................ = 1
exit(2) .................................................. WIFEXITED(2)

On Linux:

$ strace ../src/diff -N - a <&-
execve("../src/diff", ["../src/diff", "-N", "-", "a"], [/* 78 vars */]) = 0
...
ioctl(0, TCGETS, 0x7ffd75318810)        = -1 EBADF (Bad file descriptor)
fstat(0, 0x7ffd75318940)                = -1 EBADF (Bad file descriptor)
stat("a", {st_mode=S_IFREG|0664, st_size=2, ...}) = 0
open("a", O_RDONLY)                     = 0
write(2, "file_block_read fd=0 size=4096 0"..., 33file_block_read fd=0 
size=4096 0
) = 33
read(0, "a\n", 4096)                    = 2
write(2, "block_read: read(fd=0) -> 2\n", 28block_read: read(fd=0) -> 2
) = 28
read(0, "", 4094)                       = 0
write(2, "block_read: read(fd=0) -> 0\n", 28block_read: read(fd=0) -> 0
) = 28
write(2, "file_block_read block_read retur"..., 38file_block_read block_read 
returned 2
) = 38
write(2, "file_block_read fd=0 size=1 1\n", 30file_block_read fd=0 size=1 1
) = 30
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 3), ...}) = 0
write(1, "0a1\n", 40a1
)                    = 4
write(1, "> a\n", 4> a
)                    = 4
close(0)                                = 0
close(1)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++

The crucial difference is the open("a", O_RDONLY...) call, which is meant
to open the second argument. On HP-UX it returns 3; on Linux, it returns 0 -
thus making a side effect on the first argument.
In other words, on Linux we need open_safer() instead of open() - whereas
on HP-UX the open() function already behaves like open_safer() would.

So, IMO, the fix should be:
1. Include the gnulib module 'open-safer'.
2. Use open_safer() instead of open().
(Or maybe fopen_safer instead of fopen()? I haven't looked in detail.)
3. Adjust the test accordingly.

Bruno