bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS

[George Valkov]

> On 2023-02-19, at 8:08 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> 
> George, given what you've written I suppose we should give up the idea of 
> copying sparse files efficiently on macOS (and on FreeBSD 13.0-RELEASE, as it 
> has a similar bug with SEEK_HOLE and SEEK_DATA), in cases where fclonefileat 
> does not work.

I agree, Paul. I just ran some benchmarks: the time it takes to call msync 
unconditionally hurts the performance: 502 us if the memory view is already 
synced seems fine, however 28177-36585 us whenever a sync is required is more 
than the time it takes to performing a full-copy 7042-21528 us.


> Please try the attached Gnulib patch; it uses your idea of disabling 
> SEEK_HOLE and SEEK_DATA on macOS, except on steroids because it disables 
> these two macros everywhere, for all Gnulib-using applications, and it also 
> disables them on FreeBSD < 14. For coreutils you need only this patch's 
> change to lib/unistd.in.h. If this works for you I suppose we can install it 
> into Gnulib and propagate that into coreutils etc.

What is the correct way to apply your patch? I tried
git apply 0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch 
error: patch failed: ChangeLog:1
error: ChangeLog: patch does not apply
error: doc/posix-functions/lseek.texi: No such file or directory
error: lib/unistd.in.h: wrong type

Then I made the changes to lib/unistd.in.h manually, and ran the tests:
* before: corrupted
* after: fixed

Regarding FreeBSD, I have a FreeBSD 13.1-RELEASE VM running on Hyper-V, so I 
compiled and ran my samples: m.c, d.c: the copy was correct, however d.c 
reports that there are no segments to skip, which means that the file created 
by m.c using mmap is not sparse even though it should contain one blank segment 
and be sparse. Next I installed a fresh copy of FreeBSD 13.0-RELEASE and 
observed the same behaviour. Both installations use ZFS and the tests were ran 
as root.
If you know what conditions are required to reproduce the issue on FreeBSD, let 
me know and I will test it for you. Otherwise I am not sure if we need to make 
any changes to FreeBSD, since I did not observe anything wrong there.
Is it ok if I continue testing on FreeBSD 13.1, because the 13.0 I just 
installed lacks any configuration and is quite uncomfortable to use. It also 
rejects my password over SSH, so I’m forced to use the console. I can test 
pretty much anything that runs on Hyper-V.


> Also, you reported the bug against macOS 12.6. Can you check whether the same 
> bug occurs on macOS 13? If it's still there I suppose the attached patch will 
> need to be updated since it guesses the bug is fixed in macOS 13.

Yes it does. I just ran my samples under macOS Internet recovery, which reports 
as Ventura with kernel compiled on the 30th of January 2023. See the attached 
log:

ventura-internet-recovery.txt
Description: Text document

> I'll cc this email to bug-gnulib to give them a heads-up. (Gnulib readers can 
> see <https://bugs.gnu.org/61386> for context.)

Thanks, I’ll check it.


> Really, Apple should fix this serious data corruption bug in APFS and macOS.
> <0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch>

I agree, I opened a new ticket with Apple on 2023-02-16. Both via e-mail, and 
https://security.apple.com/reports/OE1928608366012

This was at the same time I reported about mmap and msync here. I got this 
replay on the following day:
> Thanks for the additional information. We are reviewing it and will follow up 
> if we have any questions or need additional details.

Feel free to write them at product-security@apple.com and include this line
OE0928602070811 - please include this ID in replies to this thread.

Introduce your self as part of GNU/coreutils and let them know it has been 18 
months since the issue was reported privately to them, so we’ve gone public 
now. CC me <same as gmail> at abv dot bg.


Georgi Valkov
httpstorm.com
nano RTOS