Rsync for backups - possible major speedup when local files renamed.

Will Smith will at willsmith.org
Thu Oct 9 01:44:01 EST 2003


I use rsync regularly for backups of large trees, sometimes
over low bandwidth links, and I would like to suggest a potentially
major speedup under some circumstances, where files or
folders are moved or renamed.

The principle is simple : often people rename a file, or move
it from one folder to another.  Current behaviour (please correct
me if I am wrong) is for wget to see this as a 'delete' and
an 'add'.  The 'add' results in a complete retransfer of the file.
This could be optimized.

For example, if I move local file 'huge_file' to 'old/huge_file',
a complete retransfer happens to the remote end of the
'new' file.


My proposed implementation is that out the outset, during generation
of file lists of local and remote, the checksums of entire files
be computed (already happening with --checksum, I believe).
Then the local and remote files are sorted by checksum and
compared.  If two files (one local, one remote) have the same
checksums but have a different filename/path on the remote
end, the remote end simply moves and renames the file
(creating folders etc as appropriate).

Would this work?  Has it been covered before? 



Thanks for your thoughts

Will Smith





More information about the rsync mailing list