problems with rsync 2.6.9 and large files (up to 20GB)

Matt McCutchen matt at mattmccutchen.net
Wed Dec 12 16:16:41 GMT 2007


On Wed, 2007-12-12 at 17:01 +0100, gorka barracuda wrote:
> but, the second time that makes this re-rsync it takes the same time
> that the first time...it seems that it doen't make an incremental
> backup with our large files... Do you think that the cause could be
> the problem of the new optimized algorithm that you put in rsync
> 3.0.0?

You are passing -I, which makes rsync transfer all regular files every
time even if they appear to be identical on source and destination.
Rsync does reduce network traffic using the delta-transfer algorithm,
but the process still rewrites each destination file in full and uses a
bunch of CPU on the sender (especially without the optimized algorithm),
so it may take a long time.  Consider whether you really need the extra
certainty of catching changes afforded by -I.  At the least, you could
disable -I for rsync runs after the first.

Matt



More information about the rsync mailing list