Re: tramp (2.1.15); Unclear doc for rsync method

[Kai Großjohann]

On 04/12/2009 11:44 AM, David Abrahams wrote:

From the TRAMP info file:

  `rsync'  --  `ssh' and `rsync'
       Using the `ssh' command to connect securely to the remote machine
       and the `rsync' command to transfer files is almost identical to
       the `scp' method.

       While `rsync' performs much better than `scp' when transferring
       files that exist on both hosts, this advantage is lost if the file
       exists only on one side of the connection.

That's pretty un-specific.  *Where*, exactly, does the file need to
exist on both machines in order to get this advantage?
  

Perhaps I wrote it and I was waving my hand because I didn't know the correct answer either :-(

Of course, on a superficial level, the answer is clear:  Suppose you execute the following sequence of commands:

rsync $REMOTE_FILE $LOCAL_FILE
emacs $LOCAL_FILE # small changes here
rsync $LOCAL_FILE $REMOTE_FILE

In this case, the second rsync command can execute faster because it only needs to transfer the small change and not the whole file.  And if tomorrow someone changes the $REMOTE_FILE, and the day after tomorrow, you repeat those three commands, and if you have kept the $LOCAL_FILE, then even the first rsync command in that sequence may run faster, for it too only needs to transfer the changes.

The more interesting question is how does this interact with Tramp?  That is the question I was trying to evade (if indeed it was me who wrote this).  I believe that it works like this:

• When you use C-x C-f to open a remote file, then Tramp will fetch the file contents to a temporary local file, and it will use a new local file name every time.  This means that there is no local file, and thus rsync cannot play out its advantage.
• When you use C-x C-s to save a remote file, then it may happen that Tramp writes to a new temporary remote file first, then renames the remote temp file to the real name, but it may also happen that Tramp writes directly to the existing remote file.  In that latter case, rsync can play out its advantage.
• Perhaps backup-by-copying and friends come into play here.
• If backup-by-copying is false, then saving a file foo means to rename foo to its backup, foo~, then to write a new file foo.  In this case, there is no advantage.
• If backup-by-copying is true, then saving a file foo means to copy foo to foo~, then overwrite the original file foo with new contents.  rsync speedup, here I come!

It's sad, really, I think.  Only saving a file can take advantage of the speedup, and only if some variables are set correctly.  We could speed up C-x C-f sometimes if Tramp were to keep a local cache of remote files.  Then we could use the cache to "prime" the transfer (by copying the cached file to the temporary file, then using rsync to update that temporary file).  I think this should be doable within the Emacs framework.

We could speed up C-x C-s if we could hook into the saving procedure, and after renaming foo to its backup foo~, we would then copy foo~ back to foo before using rsync to update its content from the local copy.  I am afraid this might not be doable within the Emacs framework.

I wonder if this is worth the effort or not.  This would only help people who use Tramp to edit large remote files on a routinely basis.  Tramp is somewhat optimized for the small-files case (by its inline transfer methods, which are required for multiple hops (unless Michael changed this)).

Kai