Speed problem

[Wayne Davison]

On Tue, Nov 12, 2002 at 04:32:31PM +0100, uwp at dicke-aersche.de wrote:
> I'd call it a bug.

No, it's not a bug.  It's the heart of the rsync algorithm at work.
Rsync trades CPU and local file I/O for network I/O in order to reduce
the amount of data that is transferred over the network.  Your diagnosis
has just shown that when the network I/O dips, rsync has traded it for
local I/O (grabbing matching blocks from the current file instead of
asking for it to be sent over the network).  For really large files that
have most matching data, most of the file I/O in building a new file
will not be network I/O, so it is to be expected that the data rate over
the net will drop when that occurs.

Note also that the --partial flag is only incidentally related to what
you were seeing since it ensured that the destination file had lots of
matching data whenever you interrupted the transfer.

The only alternative is to use the --whole-file option -- this option
turns off the rsync algorithm and just sends all the changed files over
the net completely (like an scp copy, but for changed files).  This
should only be used if you have a really fast network connection OR if
you don't want to trade the CPU and local I/O for network I/O.

..wayne..