[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

cp --reflink for SMB copy offload

From: Pádraig Brady
Subject: cp --reflink for SMB copy offload
Date: Mon, 04 Nov 2013 09:55:50 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130110 Thunderbird/17.0.2

On 11/04/2013 06:31 AM, Steve French wrote:
> On Wed, Oct 30, 2013 at 6:03 PM, Pádraig Brady <address@hidden> wrote:
>> On 10/17/2013 12:36 AM, Steve French wrote:
>>> cp --reflink opens the target file for O_WRONLY before invoking the
>>> (BTRFS) ioctl for clone file, but for copy offload over the network
>>> the SMB2 specification requires that the target file be open O_RDWR.
>>> I may be able to upgrade the target file handle on the fly by
>>> reopening it in cifs.ko, and of course I can write an SMB2/SMB3
>>> specific copy command, but it would be preferable to allow use of cp
>>> --reflink since so many people are familiar with it.
>>> There is quite a bit of flexibility in server side copy offload  -
>>> more than cp an offer, especially when using SMB3 or later dialects
>>> (e.g. in number of chunks sent at one time, chunk size, attributes
>>> copied, and even whether to use T10 style offload), but still it would
>>> be nice to support "cp --reflink" over the network.  Any ideas on
>>> this?
>>> After looking at copy.c in coreutils for cp - I couldn't think of any
>>> trivial way to force cp to open the target RW.
>>> Ideas?
>> Hi I'm one of the coreutils maintainers
>> and was behind the --reflink option to cp.
>> I'm not following the above discussion.
>> cp --reflink currently just calls the btrfs ioctl,
>> for src and dest fds, the latter which is opened O_WRONLY
>> irrespective of --reflink.
>> Are you saying that cifs is supporting this same IOCTL
>> as a high level "copy efficiently" function?
>> So you could `alias cp='cp --reflink=auto'` to copy
>> efficiently as possible?
>> There were separate reflink() and copyfile()
>> syscalls mooted recently, for which the latter would
>> be best suited to efficient backend copying on network file systems.
>> I would envisage cp trying to use copyfile() by default
>> if it ever became available.

> Couple quick comments (I would love to followup more).
> Obviously refcopy/copy-offload/server-side-copy (whatever you call it)
> - is most important for network file copies, backup, provisioning etc.
>  It is somewhat helpful for local file systems (e.g. btrfs clone does
> help improve copy speed - but no where near as much as over a network
> mount where you get an order of magnitude more savings).
> I have tested using cp --reflink over cifs mounts (actually smb2 and
> smb3 mounts using cifs.ko) with patch currently in linux-next (not
> quite complete patch series - missing some error handling, and buffer
> negotiation changes that are independent of things you would probably
> care about). - and they work fine (to most current Samba, and all
> Windows since about Windows 2003, Windows 7, Windows 8, Windows 2012
> etc. - and probably all major NAS boxes - NetApp etc are also fine).
> cp --reflink simply calls an ioctl, and easy to handle one for me (in cifs.ko)

Ok cool. I'm CC'ing address@hidden where we discuss such ideas/patches
for GNU coreutils.  Note that --reflink has different CoW semantics than
a normal copy and so would probably not be the interface used here.
I would think that cp might use some introspection as to whether
to use the "smb_copy_offload" ioctl or not, iff it had a negligible cost.

> Preliminary performance data is good.
> Ideally we should match some higher level syscalls to the behaviors
> possible with SMB2/SMB3 (since that is the most common network file
> system) - there are other copy offload features which are not exposed
> by cp --reflink

That (i.e. the copyfile() level call or equivalent I mentioned above)
would be much better and more likely to be incorporated.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]