efficient file appends

rsync at ka9q.net rsync at ka9q.net
Thu Dec 13 10:42:59 EST 2001

>While potentially a useful option, you wouldn't want the protocol to
>automatically always check for it, since it would preclude rsync on

This extension need not break any existing mechanism; if the hash of
the receiver's copy of the file doesn't match the start of the
sender's file, the protocol would continue as before.

>So at best, you'd only want to enable this option if the only thing
>for the entire set of files in a given run were files known to expand
>this way.

That's certainly reasonable, as you probably know when you issue an
rsync command whether the file in question generally changes by simple
appending. (I.e., I'd specify the option whenever I transfer a log
file or mailbox). The main drawback I can see to exchanging hashes by
default is the extra receiver CPU time consumed in producing them.

>Alternatively, even with rsync the way it is today, what I do is
>manually bump up the blocksize to something large (say 16 or 32K).

This sounds like an excellent idea, and I'll give it a try. As the
blocksize reaches the receiver's file size, the scheme essentially
approaches my idea.


More information about the rsync mailing list