Thought on large files

Matt McCutchen matt at
Wed Jan 23 06:35:55 GMT 2008

On Wed, 2008-01-23 at 13:38 +0900, Brendan Grieve wrote:
> Lets say 
> the file, whatever it is, is a 10Gb file, and that some small amount of 
> data changes in it. This is efficiently sent accross by rsync, BUT the 
> rsync server side will correctly break the hard-link and create a new 
> file with the changed bits. This means, if even 1 byte of that 10Gb file 
> changes, you now have to store that whole file again.

> What my thoughts were is that if the server could transparently break a 
> large file into chunks and store them that way, then one can still make 
> use of hard-links efficiently.

This is a fine idea, but I don't think support for this should be added
to rsync.  Instead, I suggest that you use rdiff-backup
( ), a backup tool that stores an
ordinary latest snapshot of the source along with reverse deltas for
previous snapshots and redundant attribute information both in its own


More information about the rsync mailing list