problems with rsync 2.6.9 and large files (up to 20GB)
Matt McCutchen
matt at mattmccutchen.net
Wed Dec 12 16:16:41 GMT 2007
On Wed, 2007-12-12 at 17:01 +0100, gorka barracuda wrote:
> but, the second time that makes this re-rsync it takes the same time
> that the first time...it seems that it doen't make an incremental
> backup with our large files... Do you think that the cause could be
> the problem of the new optimized algorithm that you put in rsync
> 3.0.0?
You are passing -I, which makes rsync transfer all regular files every
time even if they appear to be identical on source and destination.
Rsync does reduce network traffic using the delta-transfer algorithm,
but the process still rewrites each destination file in full and uses a
bunch of CPU on the sender (especially without the optimized algorithm),
so it may take a long time. Consider whether you really need the extra
certainty of catching changes afforded by -I. At the least, you could
disable -I for rsync runs after the first.
Matt
More information about the rsync
mailing list