Rsync help needed...
lihicks at gpi.com
Fri Mar 3 18:46:38 GMT 2006
Wayne Davison wrote:
> On Fri, Mar 03, 2006 at 09:21:25AM -0500, Linus Hicks wrote:
>> I'm transferring one file, which is obvious from my command line. Is the
>> FAQ incorrect?
> The FAQ is incomplete in how the size of the file can affect the sender's
> memory. If the destination file already exists, the sender needs to be
> able to store all the checksum data for the receiver's file in memory.
> This is nowhere near as large of a chuck of memory as a large file list,
> but I thought that perhaps this was the cause since you only experienced
> a slowdown when the receiver had a large file it was overwriting.
> One thing you don't mention is if the destination disk is local or
> network mounted. If it is not local, the extra I/O needed for the
> incremental-update algorithm could be slowing down the transfer (though
> the amount of the slowdown is very surprising). You can always choose
> to use the --whole-file option if the incremental update isn't a speed
> win for you (this is easier than having to remove the destination file
> for anything you want to update).
My post showing the first set of timings stated that all transfers are
non-local, no NFS mounts are involved. That's still true.
I am having trouble finding a case where transferring a file in incremental mode
is actually a performance win over transferring the entire file. The strength of
rsync in the cases where I use it seems to be in not transferring files based on
matching criteria like timestamp and file size. On a fast network, is there no
way to optimize an incremental transfer to out-perform a whole file transfer?
However, something seems way out of whack when it takes about four hours on a
4gb file. That I don't understand.
More information about the rsync