Thought on large files

Matt McCutchen matt at mattmccutchen.net
Thu Jan 24 05:06:00 GMT 2008


On Thu, 2008-01-24 at 13:54 +0900, Brendan Grieve wrote:
> I had a look at rdiff-backup, but I was trying to get something that
> spoke native rsync (IE, not to force any change on the client side).

To achieve this, you can have the client push to an rsync daemon and
then have the daemon call rdiff-backup so that the rdiff-backup part
happens entirely on the server.  The idea is the same as the
daemon-and-rsnapshot setup I described in the following message, but
with rdiff-backup in place of rsnapshot as the backend:

http://lists.samba.org/archive/rsync/2007-December/019470.html

> After some thought I think the best place to put such a change would
> be at the filesystem level. For example, if one had a FUSE filesystem
> that simply ran on top of an existing one, that wrote its files as I
> described (or uses diff-like methods), but presents a clean filesystem
> for rsync (or indeed any tool) to make use of. I believe I may look in
> that direction instead of hacking rsync.

You could do that, but note that the rsync receiver won't explicitly
tell the filesystem what files are similar, so you'll have to either
keep a big hashtable to help you coalesce identical blocks globally or
use some kludge like looking at what other files the receiver has open
while it is writing the destination file.

Matt



More information about the rsync mailing list