Rsync help needed...

[Linus Hicks]

Wayne Davison wrote:
> On Fri, Mar 03, 2006 at 09:21:25AM -0500, Linus Hicks wrote:
>> I'm transferring one file, which is obvious from my command line. Is the 
>> FAQ incorrect?
> 
> The FAQ is incomplete in how the size of the file can affect the sender's
> memory.  If the destination file already exists, the sender needs to be
> able to store all the checksum data for the receiver's file in memory.
> This is nowhere near as large of a chuck of memory as a large file list,
> but I thought that perhaps this was the cause since you only experienced
> a slowdown when the receiver had a large file it was overwriting.  
> 
> One thing you don't mention is if the destination disk is local or
> network mounted.  If it is not local, the extra I/O needed for the
> incremental-update algorithm could be slowing down the transfer (though
> the amount of the slowdown is very surprising).  You can always choose
> to use the --whole-file option if the incremental update isn't a speed
> win for you (this is easier than having to remove the destination file
> for anything you want to update).

My post showing the first set of timings stated that all transfers are 
non-local, no NFS mounts are involved. That's still true.

I am having trouble finding a case where transferring a file in incremental mode 
is actually a performance win over transferring the entire file. The strength of 
rsync in the cases where I use it seems to be in not transferring files based on 
matching criteria like timestamp and file size. On a fast network, is there no 
way to optimize an incremental transfer to out-perform a whole file transfer? 
However, something seems way out of whack when it takes about four hours on a 
4gb file. That I don't understand.

Linus