Re: new snapshot available: coreutils-9.2.18-ffd62.tar.xz

[Sam James]

Pádraig Brady <P@draigBrady.com> writes:

> On 13/04/2023 07:05, Sam James wrote:
>> Pádraig Brady <P@draigBrady.com> writes:
>> 
>>> On 12/04/2023 05:55, Sam James wrote:
>>>> Pádraig Brady <P@draigBrady.com> writes:
>>>>
>>>>> On 10/04/2023 22:52, Sam James wrote:
>>>>>> Sam James <sam@gentoo.org> writes:
>>>>>>
>>>>>>> Pádraig Brady <P@draigBrady.com> writes:
>>>>>>>
>>>>>>>> We plan to release coreutils-9.3 in the coming week
>>>>>>>> so any testing you can do on various different systems
>>>>>>>> between now and then would be most welcome.
>>>>>>>> This is a bug fix release coming about 3 weeks after the 9.2 release.
>>>>>>>>
>>>>>>>> --------------------------------------
>>>>>>>>
>>>>>>>> You can download the coreutils snapshot in xz format (5.7 MB) from:
>>>>>>>>        https://pixelbeat.org/cu/coreutils-ss.tar.xz
>>>>>>>>
>>>>>>>> And verify with gpg or md5sum with:
>>>>>>>>        https://pixelbeat.org/cu/coreutils-ss.tar.xz.sig
>>>>>>>>        MD5 (coreutils-ss.tar.xz) = d208d306026fb42c128a787dffcba17a
>>>>>>>>
>>>>>>>> --------------------------------------
>>>>>>>>
>>>>>>>> To test follow this standard procedure:
>>>>>>>>
>>>>>>>>        tar -xf coreutils-ss.tar.xz
>>>>>>>>        cd coreutils-9.2.18-ffd62/
>>>>>>>>        ./configure && make check VERBOSE=yes
>>>>>>>>
>>>>>>>> Failures are reported, and details are in tests/test-suite.log
>>>>>>>> Please report/attach any issues to coreutils@gnu.org
>>>>>>>>
>>>>>>>
>>>>>>> I get one failure in tests/cp/backup-dir.sh inside our packaging:
>>>>>> I see now you've caught this as a non-portable shell construct &
>>>>>> fixed -
>>>>>> thanks! So ignore this one, and see the failures below instead.
>>>>>>> ```
>>>>>>> + env cp --version
>>>>>>> [...]
>>>>>>> FAIL tests/cp/backup-dir.sh (exit status: 1)
>>>>>>> ```
>>>>>> These I can naturally still hit:
>>>>>>>
>>>>>>> In another environment (same machine but this time running manually,
>>>>>>> outside of our packaging, and on a ZFS filesystem instead of tmpfs),
>>>>>>> I get two failures:
>>>>>>>
>>>>>>> 1. tests/cp/sparse-2.sh.log: http://sprunge.us/jOCSEr (unfortunately I
>>>>>>> can't consistently reproduce this one)
>>>>>
>>>>> This might be due to slightly different I/O patterns between cp and dd.
>>>>> The log above suggests that cp is not inducing holes while dd is.
>>>>> It would be useful to get both strace files from running the following on 
>>>>> that ZFS filesystem
>>>>>
>>>>>     cd coreutils-9.2.18-ffd62
>>>>>
>>>>>     printf x>k
>>>>>     dd bs=1k seek=1 of=k count=255 < /dev/zero
>>>>>
>>>>>     strace -e cp.strace src/cp --reflink=never --sparse=always k k2
>>>>>
>>>>>     hole_size=$(stat -c %o k2)
>>>>>     strace -o dd.strace src/dd if=k of=k2.dd bs=$hole_size conv=sparse
>>>>>
>>>> By the way, this time, I also got a tests/du/basic failure too:
>>>> http://sprunge.us/xEumX6.
>>>> For the commands you gave:
>>>> ```
>>>> 255+0 records in
>>>> 255+0 records out
>>>> 261120 bytes (261 kB, 255 KiB) copied, 0.00220926 s, 118 MB/s
>>>> 2+0 records in
>>>> 2+0 records out
>>>> 262144 bytes (262 kB, 256 KiB) copied, 0.000550036 s, 477 MB/s
>>>> ```
>>>> and:
>>>> cp.strace: http://sprunge.us/cng0Z9
>>>> dd.strace: http://sprunge.us/F0Vg2r
>>>> Note that I can't reproduce the original test failure every time,
>>>> only ~1/7, 1/8 times, so I might need to run these in a loop
>>>> to get you the output needed. Or these might be fine. Dunno. :)
>>>
>>> The cp strace is interesting as it indicates the source file has holes 
>>> detected,
>>> which is unexpected. Perhaps zfs (with compression?) is detecting zeros
>>> when SEEK_DATA/SEEK_HOLE are used? The interesting part of the strace is:
>>>
>>>    lseek(3, 0, SEEK_DATA)                  = 0
>>>    lseek(3, 0, SEEK_HOLE)                  = 131072
>>>    read(3, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
>>>    write(4, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
>>>    lseek(3, 131072, SEEK_DATA)             = -1 ENXIO (No such device or 
>>> address)
>>>    ftruncate(4, 262144)                    = 0
>>>    fallocate(4, FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE, 131072, 131072) = >>> 0
>>>
>>> In any case the output pattern for dd is very similar:
>>>
>>>    read(0, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
>>>    write(1, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
>>>    read(0, "\0\0\0\0\0\0\0\0\"..., 131072) = 131072
>>>    lseek(1, 131072, SEEK_CUR)              = 262144
>>>    read(0, "", 131072)                     = 0
>>>    lseek(1, 0, SEEK_CUR)                   = 262144
>>>    ftruncate(1, 262144)                    = 0
>>>
>>> Note ZFS had this issue with SEEK_HOLE but I don't think you're hitting 
>>> that:
>>> https://github.com/openzfs/zfs/issues/11900
>>>
>> I feature heavily in that bug, too :)
>> 
>>> Given this is intermittent, I think this test may just be a little brittle,
>>> by depending on file layout which is hard to match across all file systems.
>>> The attached leverages our new --debug functionality to ensure
>>> checking for zeros is enabled.
>>>
>> tests/cp/sparse-2.log: http://sprunge.us/2W8n6b.
>> I'll pursue this with the OpenZFS folks unless you think there's
>> more we
>> can do here?
>
> I don't think ZFS is doing anything wrong here,
> just that it's a more complicated situation,
> given that it's auto "sparseifying" the initial file k.
>
> I see the issue with my test adjustment,
> as it with ZFS it may indicate "SEEK_HOLE" is being used for sparse detection,
> rather than the normal case of "zeros".
> That's a bit of a limitation of the --debug implementation actually,
> which I'll address with the first attached patch.
> With both patches attached, the test should pass now.

Yes, it does! I've tried repeatedly and it always passes, even on a cold
run (which is when it was most likely to fail before). Thanks!

I still get this du failure though:
tests/du/basic: http://sprunge.us/c5k99m

>

best,
sam

signature.asc
Description: PGP signature