[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 'cp -a' does not preserve nocow xattrs on btrfs

From: A L
Subject: Re: 'cp -a' does not preserve nocow xattrs on btrfs
Date: Sat, 5 Jun 2021 12:27:58 +0200
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2

On 2020-05-17 20:48, wrote:
"Pádraig Brady" – 17 maj 2020 kl. 13:22
On 16/05/2020 21:01, A L wrote:

On 2020-05-16 13:31, A L wrote:

Normally, 'cp -a' should preserve xattrs when copying files and folders.

 From the man page(*):
-a, --archive
same as -dR --preserve=all
preserve the specified attributes (default:
mode,ownership,timestamps), if possible additional
attributes: context, links, xattr, all

There are two important xattrs(*) on the Btrfs filesystem; 'no
copy-on-writes' ('+C') and 'compression' (+c). They have specific
conditions to work.

The nocow '+C' attribute can only be set on empty files.
The compression '+c' can be set on any file, but only newly written
data will be compressed.

The problem is that 'cp -a' seems to set the xattrs after creating the
file and writing the data to it, which means that the nocow '+C' flags
can't be set and that files would not be fully compressed, even though
the flag is set.

I think that these xattrs should be preserved when doing 'cp -a'.


I realise I had confused the difference between fattr (aka ext2
attributes) and xattrs (extended attributes). In Btrfs, the nocow '+C'
is only available as fattr, and not a xattr, which is why it is not
copied with cp. Apart from that, the issue still stands with compression
flag '+c', since that is also implemented as an xattr
("btrfs.compression") . cp ought to set the xattrs before writing any
data to the file, possibly with an fsync in between.

Side note: Why is it that cp -a does not copy file attrs and only
xattrs? As a end user, the naming similarity fattr and xattr is somewhat
confusing, and the man page does not clearly state that only the latter
is supported in cp.

Seems the same flags interface is used by ext4, btrfs, and xfs at least:
Given the generality, it's plausible we might consider preserving these flags.

Copying all xattrs before writing the data may impact the ability to write data,
so we may need to split which xattrs are written when. Though if we
just considered the fattrs it might simplify (though we may have the same issue 

As for the particular NOCOW flag, is that something you'd always want
to set on new files? It seems like it might be more specific to
the file being written, than the file being read?
This would overlap with the existing cp --reflink flag.
Perhaps cp --reflink=never would preserve the NOCOW flag.
one can have NOCOW set on a dir, which causes each file to be NOCOW.
one can't reflink a NOWCOW file

thanks for the info,

Perhaps it is best to limit my request to fattrs for the moment. I don't know 
how well defined the xattrs are in general.

Looking at the list of available fattrs, I think only the immutable attribute 
can't be set first. There are some attributes that may never be set with 
chattr. Not sure how to handle those.

How should cp react when the target fs does not support one or more attributes?

Regarding nocow. I think all fattrs should only be preserved with -a or 
--preserve=fattr (or all).if you also specify reflink=always/auto then perhaps 
the -a flag should have priority?


Hi again,

Why not make a rulebook for when attrs should be set during cp -a? IMHO the intent of 'cp -a' is to preserve all source attributes, if possible.

On Btrfs, nocow (+C) files can be reflinked too, but only to other nocow files. Thus, in order to enable 'cp -a --reflink=always' for nocow files, we need to set '+C' on the destination files before we add actual file data to them.

Here is an example where a nocow file is reflinked. It works around 'cp -a' by setting +C on the target dir first.

  # mkdir foo bar
  # touch foo/file
  # chattr +C foo/file
  # chattr +C bar
  # cat somefile >> foo/file
  # cp -a --reflink --sparse=auto foo/file bar/file
  (on my distro, cp complained about reflink if sparse=auto was not set)

  # compsize foo bar
  Processed 2 files, 1 regular extents (2 refs), 0 inline.
  Type       Perc     Disk Usage   Uncompressed Referenced
  TOTAL      100%       12K          12K          24K
  none       100%       12K          12K          24K

Can we generally say that all flags except ‘i’ immutable can and should be set before filedata is added to new files and that directories should have flags set _after_ all containing files have been added?

a       Order does not matter.
A       Order does not matter.
c On empty file, before data is added. Otherwise file content won’t be compressed.
C       Must be set on empty file, before data is added or reflinked.
d       Order does not matter.
D       Only valid dirs. Does order matter?
i       Must be set after file has been written to.
S       Does order matter?
m On empty file, before data is added. Otherwise file content could be compressed due to mount flags or inheritance from parent directory flags.

There are other flags not supported by Btrfs, which may have other requirements?

# Currently supported Btrfs file attributes:

a       append only, new writes are always written at the end of the file

A       no atime updates

c compress data, all data written after this attribute is set will be compressed. Please note that compression is also affected by the mount options or the parent directory attributes. When set on a directory, all newly created files will inherit this attribute.

C       no copy-on-write, file modifications are done in-place
When set on a directory, all newly created files will inherit this attribute. Note: due to implementation limitations, this flag can be set/unset only on empty files.

d no dump, makes sense with 3rd party tools like dump(8), on BTRFS the attribute can be set/unset but no other special handling is done

D synchronous directory updates, for more details search open(2) for O_SYNC and O_DSYNC

i immutable, no file data and metadata changes allowed even to the root user as long as this attribute is set (obviously the exception is unsetting the attribute)

S synchronous updates, for more details search open(2) for O_SYNC and O_DSYNC

m (previously X) no compression, permanently turn off compression on the given file. Any compression mount options will not affect this file. When set on a directory, all newly created files will inherit this attribute.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]