Re: new snapshot available: coreutils-9.2.18-ffd62.tar.xz

[Pádraig Brady]

On 12/04/2023 05:55, Sam James wrote:
> Pádraig Brady <P@draigBrady.com> writes:
>
> > On 10/04/2023 22:52, Sam James wrote:
> > > Sam James <sam@gentoo.org> writes:
> > >
> > > > Pádraig Brady <P@draigBrady.com> writes:
> > > >
> > > > > We plan to release coreutils-9.3 in the coming week
> > > > > so any testing you can do on various different systems
> > > > > between now and then would be most welcome.
> > > > > This is a bug fix release coming about 3 weeks after the 9.2 release.
> > > > >
> > > > > --------------------------------------
> > > > >
> > > > > You can download the coreutils snapshot in xz format (5.7 MB) from:
> > > > >       https://pixelbeat.org/cu/coreutils-ss.tar.xz
> > > > >
> > > > > And verify with gpg or md5sum with:
> > > > >       https://pixelbeat.org/cu/coreutils-ss.tar.xz.sig
> > > > >       MD5 (coreutils-ss.tar.xz) = d208d306026fb42c128a787dffcba17a
> > > > >
> > > > > --------------------------------------
> > > > >
> > > > > To test follow this standard procedure:
> > > > >
> > > > >       tar -xf coreutils-ss.tar.xz
> > > > >       cd coreutils-9.2.18-ffd62/
> > > > >       ./configure && make check VERBOSE=yes
> > > > >
> > > > > Failures are reported, and details are in tests/test-suite.log
> > > > > Please report/attach any issues to coreutils@gnu.org
> > > >
> > > > I get one failure in tests/cp/backup-dir.sh inside our packaging:
> > > I see now you've caught this as a non-portable shell construct &
> > > fixed -
> > > thanks! So ignore this one, and see the failures below instead.
> > > > ```
> > > > + env cp --version
> > > > [...]
> > > > FAIL tests/cp/backup-dir.sh (exit status: 1)
> > > > ```
> > > These I can naturally still hit:
> > > > In another environment (same machine but this time running manually,
> > > > outside of our packaging, and on a ZFS filesystem instead of tmpfs),
> > > > I get two failures:
> > > >
> > > > 1. tests/cp/sparse-2.sh.log: http://sprunge.us/jOCSEr (unfortunately I
> > > > can't consistently reproduce this one)
> >
> > This might be due to slightly different I/O patterns between cp and dd.
> > The log above suggests that cp is not inducing holes while dd is.
> > It would be useful to get both strace files from running the following on that 
> > ZFS filesystem
> >
> >    cd coreutils-9.2.18-ffd62
> >
> >    printf x>k
> >    dd bs=1k seek=1 of=k count=255 < /dev/zero
> >
> >    strace -e cp.strace src/cp --reflink=never --sparse=always k k2
> >
> >    hole_size=$(stat -c %o k2)
> >    strace -o dd.strace src/dd if=k of=k2.dd bs=$hole_size conv=sparse
>
> By the way, this time, I also got a tests/du/basic failure too:
> http://sprunge.us/xEumX6.
>
> For the commands you gave:
> ```
> 255+0 records in
> 255+0 records out
> 261120 bytes (261 kB, 255 KiB) copied, 0.00220926 s, 118 MB/s
> 2+0 records in
> 2+0 records out
> 262144 bytes (262 kB, 256 KiB) copied, 0.000550036 s, 477 MB/s
> ```
>
> and:
> cp.strace: http://sprunge.us/cng0Z9
> dd.strace: http://sprunge.us/F0Vg2r
>
> Note that I can't reproduce the original test failure every time,
> only ~1/7, 1/8 times, so I might need to run these in a loop
> to get you the output needed. Or these might be fine. Dunno. :)

The cp strace is interesting as it indicates the source file has holes detected,
which is unexpected. Perhaps zfs (with compression?) is detecting zeros
when SEEK_DATA/SEEK_HOLE are used? The interesting part of the strace is:

  lseek(3, 0, SEEK_DATA)                  = 0
  lseek(3, 0, SEEK_HOLE)                  = 131072
  read(3, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
  write(4, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
  lseek(3, 131072, SEEK_DATA)             = -1 ENXIO (No such device or address)
  ftruncate(4, 262144)                    = 0
  fallocate(4, FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE, 131072, 131072) = 0

In any case the output pattern for dd is very similar:

  read(0, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
  write(1, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
  read(0, "\0\0\0\0\0\0\0\0\"..., 131072) = 131072
  lseek(1, 131072, SEEK_CUR)              = 262144
  read(0, "", 131072)                     = 0
  lseek(1, 0, SEEK_CUR)                   = 262144
  ftruncate(1, 262144)                    = 0

Note ZFS had this issue with SEEK_HOLE but I don't think you're hitting that:
https://github.com/openzfs/zfs/issues/11900

Given this is intermittent, I think this test may just be a little brittle,
by depending on file layout which is hard to match across all file systems.
The attached leverages our new --debug functionality to ensure
checking for zeros is enabled.

> > > > 2. ./tests/misc/tty-eof.log: http://sprunge.us/yWNByM
> >
> > perl's expect module seems to be getting spurious exitstatus for some commands.
> > I suspect a race, and there is a hardcoded 10s in that script.
> > There are various other issues in that script actually.
> > Hopefully the attached addresses this.
>
> Interestingly, it doesn't!
>
> tests/misc/tty-eof.log: http://sprunge.us/UwqooK.
>
> I just applied the patch (and verified the changes were there) and
> ran 'make check' again rather than a rebuild, but I assume that
> shouldn't matter given it's a test script change.

Drats. It's awkward to debug further when I can't repro.
Given it happens to random commands, I still suspect an issue with your Expect 
lib,
or with how we're using it, rather than an issue with the commands themselves.

cheers,
Pádraig

coreutils-sparse-2-fix.patch
Description: Text Data