Re: coreutils-6.2: various runtime problems on Darwin-8.7.0 HFS+ (including attachment this time)

[Jim Meyering]

<address@hidden> wrote:
> I am new to this list, but not at all new to building and using
> open source projects.  (Please CC me if needed as I haven't yet
> joined this list.  Or I think I can access and post via
> news.gmane.org with regular newsreaders.)
>
> I built coreutils-6.2 on Darwin-8.7.0 ("Tiger" 10.4.7 and the
> latest XCode-2.4).  I actually have installed coreutils-5.97 to
> compare with.
>
> 5.97 seems to work fine, but 6.2 seems to have many things wrong.
>
> Most notably the new 'ls' in 6.2 is acting very very strange.
> We get double listings in 'long' mode of every file, with
> exactly the first set saying "No such file or directory",
> including the dot-dirs.  The second set lists each file/dir as
> expected.
>
> I am attaching a sample to show you what I mean here.  Please
> find the archive named 'coreutils-6.2-bugs.tar.bz2'.  It should
> expand to create these files:
>
> -rw-r--r-- 1 root wheel 1957313 Sep 25 00:43 config.log
> -rw-r--r-- 1 root wheel   31244 Sep 25 00:43 config_consolelog.txt
> -rw-r--r-- 1 root admin   29725 Sep 25 00:51
> ls_whoops_consolelog.txt
> -rw-r--r-- 1 root wheel   67706 Sep 25 00:45 make_consolelog.txt
> -rw-r--r-- 1 root wheel   55908 Sep 25 00:49
> makecheck_consolelog.txt

Thank you for the complete bug report!

> I'm obviously quite worried about releasing this code to Darwin
> users, so I thought I better do a bit of hollarin' here.  ;)

You're right to be cautious.
The problems involving ls, cp, and install are all due
to the fact that Darwin's acl_get_file function (used
in coreutils' lib/acl.c) is not behaving as advertised:
it's returning NULL and setting errno to ENOENT for a file
that *does* exist.  However, the man page says it does that only
for files that do not exist.  When a function like that reports
a failure, coreutils programs report it and exit nonzero, and
that's why you're seeing so many diagnostics and test failures.

Using the very latest version from coreutils and gnulib,
when I turn off ACL support altogether (comment out the USE_ACL
definition in lib/config.h), all of the tests pass except tests/dd/misc.

That failure appears to be due to another Darwin-specific bug:
The open flag, O_NOFOLLOW is defined, yet open doesn't honor the
semantics we've come to expect for that flag.  That makes the test
of dd's new iflag=nofollow option fail.
The only mitigating factor is that O_NOFOLLOW is not yet documented in
their open man page:  the definition may have slipped into their
system headers before library support was completed.

Unfortunately, O_NOFOLLOW (when present) is an important key to avoiding
certain types of race conditions.  Defining the symbol yet letting
open ignore it is *bad*.  Here's a small program to demonstrate the bug.
Run the command in the comment.  If it prints "bug", then your system's
open function is buggy.
----------------
$ cat o_nofollow.c
/* rm -f s f; cc o_nofollow.c && { touch f; ln -s f s; ./a.out || echo bug; } */
# define _GNU_SOURCE 1
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
int
main ()
{
  int fd = open ("s", O_RDONLY|O_NOFOLLOW);
  return 0 < fd;
}
----------------

However, the good news is that this led me to discover a bug in gnulib's
test for just this type of brokenness.  Its fcntl_h module tests for a
nonfunctional O_NOFOLLOW, but the test had a typo.  I've just checked
in a fix for that.

So now, all tests now pass, using the very latest, and with the hack
to disable ACL support.  I suppose we'll need an added run-test
to detect the acl_get_file brokenness.