Re: new snapshot available: coreutils-9.2.18-ffd62.tar.xz

[Sam James]

Pádraig Brady <P@draigBrady.com> writes:

> On 12/04/2023 05:55, Sam James wrote:
>> Pádraig Brady <P@draigBrady.com> writes:
>> 
>>> On 10/04/2023 22:52, Sam James wrote:
>>>> Sam James <sam@gentoo.org> writes:
>>>>
>>>>> Pádraig Brady <P@draigBrady.com> writes:
>>>>>
>>>>>> We plan to release coreutils-9.3 in the coming week
>>>>>> so any testing you can do on various different systems
>>>>>> between now and then would be most welcome.
>>>>>> This is a bug fix release coming about 3 weeks after the 9.2 release.
>>>>>>
>>>>>> --------------------------------------
>>>>>>
>>>>>> You can download the coreutils snapshot in xz format (5.7 MB) from:
>>>>>>       https://pixelbeat.org/cu/coreutils-ss.tar.xz
>>>>>>
>>>>>> And verify with gpg or md5sum with:
>>>>>>       https://pixelbeat.org/cu/coreutils-ss.tar.xz.sig
>>>>>>       MD5 (coreutils-ss.tar.xz) = d208d306026fb42c128a787dffcba17a
>>>>>>
>>>>>> --------------------------------------
>>>>>>
>>>>>> To test follow this standard procedure:
>>>>>>
>>>>>>       tar -xf coreutils-ss.tar.xz
>>>>>>       cd coreutils-9.2.18-ffd62/
>>>>>>       ./configure && make check VERBOSE=yes
>>>>>>
>>>>>> Failures are reported, and details are in tests/test-suite.log
>>>>>> Please report/attach any issues to coreutils@gnu.org
>>>>>>
>>>>>
>>>>> I get one failure in tests/cp/backup-dir.sh inside our packaging:
>>>> I see now you've caught this as a non-portable shell construct &
>>>> fixed -
>>>> thanks! So ignore this one, and see the failures below instead.
>>>>> ```
>>>>> + env cp --version
>>>>> [...]
>>>>> FAIL tests/cp/backup-dir.sh (exit status: 1)
>>>>> ```
>>>> These I can naturally still hit:
>>>>>
>>>>> In another environment (same machine but this time running manually,
>>>>> outside of our packaging, and on a ZFS filesystem instead of tmpfs),
>>>>> I get two failures:
>>>>>
>>>>> 1. tests/cp/sparse-2.sh.log: http://sprunge.us/jOCSEr (unfortunately I
>>>>> can't consistently reproduce this one)
>>>
>>> This might be due to slightly different I/O patterns between cp and dd.
>>> The log above suggests that cp is not inducing holes while dd is.
>>> It would be useful to get both strace files from running the following on 
>>> that ZFS filesystem
>>>
>>>    cd coreutils-9.2.18-ffd62
>>>
>>>    printf x>k
>>>    dd bs=1k seek=1 of=k count=255 < /dev/zero
>>>
>>>    strace -e cp.strace src/cp --reflink=never --sparse=always k k2
>>>
>>>    hole_size=$(stat -c %o k2)
>>>    strace -o dd.strace src/dd if=k of=k2.dd bs=$hole_size conv=sparse
>>>
>> By the way, this time, I also got a tests/du/basic failure too:
>> http://sprunge.us/xEumX6.
>> For the commands you gave:
>> ```
>> 255+0 records in
>> 255+0 records out
>> 261120 bytes (261 kB, 255 KiB) copied, 0.00220926 s, 118 MB/s
>> 2+0 records in
>> 2+0 records out
>> 262144 bytes (262 kB, 256 KiB) copied, 0.000550036 s, 477 MB/s
>> ```
>> and:
>> cp.strace: http://sprunge.us/cng0Z9
>> dd.strace: http://sprunge.us/F0Vg2r
>> Note that I can't reproduce the original test failure every time,
>> only ~1/7, 1/8 times, so I might need to run these in a loop
>> to get you the output needed. Or these might be fine. Dunno. :)
>
> The cp strace is interesting as it indicates the source file has holes 
> detected,
> which is unexpected. Perhaps zfs (with compression?) is detecting zeros
> when SEEK_DATA/SEEK_HOLE are used? The interesting part of the strace is:
>
>   lseek(3, 0, SEEK_DATA)                  = 0
>   lseek(3, 0, SEEK_HOLE)                  = 131072
>   read(3, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
>   write(4, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
>   lseek(3, 131072, SEEK_DATA)             = -1 ENXIO (No such device or 
> address)
>   ftruncate(4, 262144)                    = 0
>   fallocate(4, FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE, 131072, 131072) = 0
>
> In any case the output pattern for dd is very similar:
>
>   read(0, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
>   write(1, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
>   read(0, "\0\0\0\0\0\0\0\0\"..., 131072) = 131072
>   lseek(1, 131072, SEEK_CUR)              = 262144
>   read(0, "", 131072)                     = 0
>   lseek(1, 0, SEEK_CUR)                   = 262144
>   ftruncate(1, 262144)                    = 0
>
> Note ZFS had this issue with SEEK_HOLE but I don't think you're hitting that:
> https://github.com/openzfs/zfs/issues/11900
>

I feature heavily in that bug, too :)

> Given this is intermittent, I think this test may just be a little brittle,
> by depending on file layout which is hard to match across all file systems.
> The attached leverages our new --debug functionality to ensure
> checking for zeros is enabled.
>

tests/cp/sparse-2.log: http://sprunge.us/2W8n6b.

I'll pursue this with the OpenZFS folks unless you think there's more we
can do here?

>>>>> 2. ./tests/misc/tty-eof.log: http://sprunge.us/yWNByM
>>>
>>> perl's expect module seems to be getting spurious exitstatus for some 
>>> commands.
>>> I suspect a race, and there is a hardcoded 10s in that script.
>>> There are various other issues in that script actually.
>>> Hopefully the attached addresses this.
>> Interestingly, it doesn't!
>> tests/misc/tty-eof.log: http://sprunge.us/UwqooK.
>> I just applied the patch (and verified the changes were there) and
>> ran 'make check' again rather than a rebuild, but I assume that
>> shouldn't matter given it's a test script change.
>
> Drats. It's awkward to debug further when I can't repro.
> Given it happens to random commands, I still suspect an issue with your 
> Expect lib,
> or with how we're using it, rather than an issue with the commands themselves.

No worries - I'll see if I can dig into it, I'm not optimistic though
because others with a similar environment can't seem to hit it either.

I'll let you know if I do get anywhere though.

Thank you!

>
> cheers,
> Pádraig
>
best,
sam

> [2. text/x-patch; coreutils-sparse-2-fix.patch]...

signature.asc
Description: PGP signature