Re: [Bug-tar] Detection of sparse files is broken on btrfs

[Mark H Weaver]

Hi Joerg,

Joerg Schilling <address@hidden> writes:

> Paul Eggert <address@hidden> wrote:
>
>> On 01/08/2018 08:06 AM, Joerg Schilling wrote:
>> > blkcnt_t st_blocks      Number of blocks allocated for this object.
>> >
>> > I hope I do not need to explain the term "allocated".
>>
>> I'm afraid that you do need to explain "allocated". Suppose, for 
>> example, two files are clones: they have different inode numbers and are 
>> different files from the POSIX point of view, but they have the same 
>> contents and only one copy exists at the lower level. How many blocks 
>> are "allocated" for each file?
>
> POSIX does not explain what happens with several filesystems.
>
> For this reason, I cannot see any reason why deduplication between different 
> filesystems that share a common dataset should be alllowed to be visible at 
> user level.
>
> Also note that "du" may report more blocks than you expect in case it does not
> have the ability to honor hard linked files.
>
> The most important fact however is that allocating spade happens before you 
> copy data into that space. A file with more data than what can be hold in the 
> inode thus must have st_blocks > 0.

You're making a lot of assertions about how you believe systems should
behave, but so far, I've seen only one vague line quoted from POSIX that
falls far short of backing up your assertions.

In any case, even if you were able to show that POSIX clearly prohibits
the Btrfs behavior (and so far you've failed to convince even the people
on this list) the question of whether Linux conforms to POSIX is rather
academic, and not particularly relevant.

The fact is, Linux (the kernel) is by far the most popular POSIX-like
kernel, and if GNU Tar loses user data because it makes assumptions that
simply do not hold in the real world, people are going to blame us, not
the Linux developers.

Somehow, we need to make GNU Tar and Linux (including GNU Linux-Libre)
work well together.  If they don't, I promise you that it is GNU Tar
that will be dumped (or more likely forked), not Linux.

If this st_blocks issue is truly important to you, then you need to
either convince the Linux developers to adopt your ideas of how things
should work, or else you need to convince users to boycott Linux and
switch to another kernel.  Good luck with that.

What we mustn't do is to bury our heads in the sand and pretend that
everything conforms to your idea of how kernels should behave.

If a user comes to us and complains that they lost important data
because of this issue that was reported to us in 2016, do you really
think your arguments are going to hold up?

I don't expect any of this to convince you, but it is most likely the
last message I will write in this "debate" between you and the rest of
the world.  Instead, I will focus on fixing the bug.

      Mark